Keynote (TensorFlow Dev Summit 2018)

Key Takeaways

[Music] good morning everybody I'm Anita Vijay Kumar technical program manager for tensorflow welcome good morning to all of you it's really exciting to be here with all of you today and welcome to the 2018 tensorflow developer summit we have a packed day filled with lots of interesting talks and some really cool demos we hope you enjoy it as much as we do as you know we are together embarking on a transformative journey in the field of machine learning and artificial intelligence problems that were impossible or too complex are now possible to solve using this technology this field is unleashing new frontiers and substantially changing many aspects of our lives in a meaningful way we've already seen great advantages in many different areas with machine learning some examples are in the field of astronomy hunting for new planets is no easy task these planets are small cold dark objects compared to their host star researchers and astrophysicists have analyzed large amounts of data from the Kepler mission and by using machine learning have discovered a brand-new planet called the kepler 90i making it the only eight planet solar system that we know of in our universe other than our own in healthcare we have discovered that looking at scans of the human eye machine learning can assess a person's risk for cardiovascular diseases just imagine if the screening can be done with a mobile device how profound the effect will be suddenly the whole world will have access to rapid easy affordable non-invasive screening for heart diseases saving lots of lives in aviation tradition it's logic tree prediction is a critical component to ensuring that era aircrafts can fly safely and efficiently in crowded spaces Engineers and air traffic controllers in Europe are using machine learning to predict the trajectory of a flight through the air spaces of Belgium Luxembourg Germany and the Netherlands this has more than 1.8 million flights per year and is one of the most dense and complex air spaces in the world in dairy farming we know that a cow's health is vital to the survival of the dairy industry and folks that connector our company in the Netherlands wondered if they can use machine learning to track the health of cows and being able to provide insights to farmers and veterinarians on actions to be taken to ensure we have happy healthy cows that are high yielding so we now have happy comes not only from California but also the Netherlands fighting climate change fighting climate change and production in protecting our forests we believe that technology has a huge role to play a group of engineers and scientists have built real-time detection and alerting mechanisms to detect logging of trees they listen to change they listen to real time data foot from chainsaw sounds from talking sounds from logging sounds and machine learning is fighting deforestation and protecting our precious rainforests in music there are many machine learning algorithms using deep neural networks that can learn the characteristics of sounds and create completely new music based on these characteristics in arts and culture you can even search if you have a long-lost twin from the past by looking at oil paintings that are based using apps that are based on machine learning how cool is that at Google we are an AI first company and every product is being fundamentally impacted and changed by machine learning beat the popular Google home or the pixel or search or YouTube or even Maps do you know what is fascinating in all these examples tensorflow is at the forefront of them making it all possible a machine learning platform that can solve challenging problems for all of us join us on this incredible journey to make tensorflow powerful scalable and the best machine learning platform for everybody I now invite Rajat director of engineering of tensorflow to tell us more about this thank you hello everyone good morning thanks Anita so let's take a look at what we've been doing over the last two years it's been really amazing with lots of new releases if you've seen the popularity of pencil or of tensorflow really grow over the last two years especially over the last year we've focused on making tensorflow easy to use and with release of high-level api's you know like TF data in TF Carris and new programming paradigms like a you go execution really continue to make that easier earlier this year we hit the milestone of 11 million downloads you know really excited to see how much users are using this in how much impact it's having in the world so here's a map showing self-identified locations of folks on github that have starred tensorflow and you can see that it goes all the way up to the Arctic Circle it room so in Norway in all the way south down to Deception Island which is part of Antarctica in fact tensorflow is used in every single time zone in the world an important part of any open-source project are the contributors themselves you know the community is made of the people who make this project successful I'm excited to see over a thousand contributors from outside Google who are making contributions not just by improving code but also by helping the rest of the community by answering questions responding to queries and so on our commitment to this community is by sharing our direction in a roadmap inviting public participation on design direction and by creating SIG's that help focus collaboration on the key needs like build in packaging tents aboard and we'll be talking more about this in detail later this afternoon today we are launching a new tensorflow blog will be sharing work by the team and the community on this blog and we would like to invite you to participate in this as well we are also launching a new YouTube channel for tensorflow that brings together all the great content that's been created for tensorflow again all of these are for the community to really help build and communicate all day today we will be sharing a number of posts on the blog and videos on the channel the talks you're hearing here will be made available there as well along with lots of conversations and interviews with the speakers to make reuse and sharing easier today we are launching tensorflow hub this library of model components is easy to integrate into your models with just a single line of code you know again goes back to really making things easy for you machine learning is important and powerful as you saw we care about bringing it to more and more people as part of this effort we recently released the machine learning crash course focused on introducing this to a much broader audience the same course that strained thousands of Googlers and machine learning and allows us to integrate ml into everything we build is now even available to everyone tensorflow started as a new library for numerical computation with a focus on deep learning and neural networks since the early release it's grown to include a very rich collection of machine learning algorithms it includes popular algorithms like linear regressions and decision trees commonly used for many structured data classification problems there's also a very broad collection of state-of-the-art tools for stats and Bayesian analysis in the TF probability library and several new additions are being released today you can check out the blog post in website for more details as I mentioned earlier one of the big key focus points for us is to make tensorflow easy to use and towards that we've been pushing on simpler API is easier to use and making them more intuitive at the lowest level you know our focus is to consolidate a lot of the api's we have and make it easier to build these models and train them at the lowest level the tensorflow API is are really really flexible and they let users build anything they want to with either execution the same API zarnow easier to use tensorflow also contains a full implementation of cares TF cares contains building blocks like layers to build models to make it easier to do that and offers lots of utilities to train them as well caris works with both Graff and eager execution as well for distributed execution we provide estimators so you can take your models and combine them and estimators to distribute them across machines you can also build estimators from the chaos layers or models and finally we provide pre-made estimators a library of ready to go implementations of common machine learning algorithms so let's take a look at how this works so first you would often define your model and this is a nice and easy way to define your model she was a convolutional model here with just a few lines here now once you've defined that often you want to do some input processing so in this case you load your data set from files and we have a great API TF data that was introduced in 1.4 that makes it easy for you to process inputs while still allowing us to do lots of optimizations behind the scenes and you'll see a lot more detail on this later today as well once you have those the model and the input data and now you can put them together by iterating over the data set computing gradients and updating the parameters themselves as you can see here you really just need a few lines of Python to put these together and in fact it runs just as Python does and you can use your Python debugger to debug that and solve problems as well of course you can do it in even fewer lines by just using the predefined utilities that we have at Kara's in this case it executes the model as a graph with all the additional optimizations that come with it now this is great for a single machine or a single device now often given the high heavy computation needs for deep learning or machine learning we want to use more than one accelerator or GPU or our machines themselves so for that we have estimators making the same model that you define the same data sets that you had you can can build an estimator and really use that to Train across this cluster or across the multiple devices on a single machine that's great why not use a cluster why use just a single blob box if you can actually train faster with that this picture shows a TPU part used for training ml models at scale and again the focus is to really take everything you've been doing and just you know build a teepee your estimator that allows you to scale that same model across this entire supercomputer and then finally once you've trained that model you can use that one line at the bottom to export that for deployment itself so you know deployment is important you often do that in data centers but more and more we're seeing the need to deploy this on the phones on other devices as well and so for that we have tens of lowlight the model that we just exported we have a script to really convert that to a custom format that's designed for devices that's lightweight and it's really fast to get started with and then once you have that format you can include that in your application integrate tensorflow Lite with a few lines here and get going you basically have an application that can do ML predictions and include ml and whatever tasks you want to perform so tensile flow runs not just in many platforms but in many languages as well today I'm excited to add Swift to the mix and it brings with it a fresh approach to machine learning don't miss the talk by Chris Lackner this afternoon that covers the exciting details of how we are envisioning this platform javascript is a language that's synonymous with the web development community I'm excited to announce tensor flora Jas it's a it's bringing machine learning to this growing community of fix web developers so let's take a brief look at this brings the same tensorflow operations that you're used to into JavaScript where you can call them just as plain JavaScript code and on with the full-fledged layers API on top it also includes full support for tensorflow and caris models so you can pick the most appropriate development and deployment environment that works for you so you can start in one place go to the other whichever way you want and none of the covers these api's are accelerated in the browser via WebGL and then the node.js support is coming soon which will give you the power of all of tensorflow tax rate on CPUs and GPUs as well and with that I would like to invite Megan Cachola our engineering director Google brain who leads our performance efforts to talk a bit more about how tensorflow does performance [Music] thanks ratchet so performance across all platforms is critical to tensorflow success first I want to take a quick step back and just talk about some of the things that we think about when we're measuring and assessing intent flows performance one of the things we try to do is focus on real-world data and time to accuracy it's really important to have reproducible benchmarks we want to make sure those benchmarks are actually realistic of the workloads and types of things that users like you are actually doing on a day to day basis another thing like Rajat talked about is that we want to make sure we have clean api's and we don't want to have like a fast version and a pretty version of something the fast version is the pretty version and so all of these api's that we're talking about like Rajat is mentioned that we're going to be talking about through various talks throughout the day today these are the things you can use to get the best performance out of tensorflow so that you don't have to worry about like which one is fast which one is pretty use the pretty one it is fast you'll hear about TF data from Derek right after the keynote as well as distribution strategy from Egor later today and these are just some great examples of things we've been pushing on to ensure good performance and clean api's so in general we want to ensure that you have great performance regardless of what type of platform you might be making use of whether it's a large data center like what's shown here or maybe you're using something like what's shown on the image here you have a GPU or a CPU box under your desk maybe you're making use of a cloud platform or mobile or an embedded device regardless of the type of hardware you're making you so we want to make sure that tensorflow performs well across all of them so now getting to some numbers because what is the performance talk if I don't show you slides and numbers so first let's look at things on the mobile side so this is highlighting tensorflow light performance so tons of flow light we initially release last fall and there's going to be a talk giving a lot more detail just about how it works and the things we were thinking of when making it later today by Sarah but I wanted to call out just here the speed ups that we see in inference so a 3x inference speed-up when using tensorflow like quantized versus standard tensorflow light for some of the models called out here and again it's critical to have strong performance regardless of what platform you're working on and we're really excited to be able to see these gains in mobile in looking past mobile just beyond there are a number of companies in the hardware space which continues to expand the contributions that come out of the collaborations that we have with these companies the contributions they give back to tensorflow and back to the community at large are critical to making sure that tensorflow performs well on these specific platforms for the users that each group really cares about one of the first ones I want to highlight is Intel so there's the Intel MKL DNN library which is open sourced and it's highly optimized for tensorflow so by making use of this open source library were able to achieve a 3x inference speed-up on various Intel platforms I've called out Broadwell and skylake here as well as the great scaling efficiency on training and again this is one of those things that really highlights how important it is to have strong collaborations with different folks in the community and we're really excited to be able to see things like this they can go back to all the users I wanted to call out a few of the collaborations we've had with Nvidia as well so one of the first things that we're excited about is tensor RT sensor RT is an inference optimizer and a runtime for GPU platforms we've been collaborating very closely with Nvidia on tensor RT 10 so our T's been around for a little while but with the tensorflow 1.7 release we now have native support for tensor RT built in and that means you can get low latency high throughput and friends so here you can see an 8x inference speed-up when making use of tensor RT versus just using native FP 32 with standard tensorflow so it's great again to see these types of collaborations that can be done and the contributions come back and the great numbers that can be delivered by it looking past inference and going on to some of the training things so mixed precision training is really important as faster and more powerful Hardware comes out some of it is optimized such that if you use mixed precision or FP 16 support that's how you'll get the best out of that hardware one example of this is the Tesla V 100 that NVIDIA has so we've been working closely to make sure that we have this strong mix mix precision training support so that way you can get the best performance out of that hardware so here you can see a 3.5 X training speed up so this is on an 8 X Tesla view 100 bucks and you can see just the performance improvements when you move to mix precision training versus just using the standard tensorflow scaling efficiency is really important as well you know obviously we want to make sure tentacle performs well maybe you have a single GPU but it should keep going regardless of how many things that you throw at it so we want to make sure that again looking at examples for real-world data as well as synthetic data it's great to benchmark on synthetic data but it's very important to make sure we'll use cases actually perform as we expect as well so here we're showing 90 percent scaling efficiency with real data ninety-five percent with synthetic data and this is again on DG x1 so an Nvidia box that has a V 100 box that has between 1 and 8 GPUs and you can kind of see the scaling here but this is something that we care a lot about and you're actually going to hear more about how to get easier scaling efficiency by making use of some of our internal API is later today as well moving past moving on to cloud frameworks I want to talk a little bit about cloud CPUs so cloud CPUs was launched in beta in February so just a month and a half ago this is Google's v2 TPU so it's designed from the ground up for machine learning more clothes and it's available via a Google cloud platform like I mentioned it's really exciting to look at some of the numbers here so this picture is actually showing a device and on a single device you're able to get a hundred and eighty teraflops of computation but it's not just about the compute power it's about what you can actually do on this right it doesn't really if you have amazing compute power but you can't run the types of models that you want so I want to highlight the types of models that reference models that we've already open-source that are available today as well as a bunch more types of models that are coming soon again just to highlight the breadth of things that you can run on this type of hardware and get great performance as well as great accuracy we actually have an internal team that is continually making sure that these models perform well they perform fast and that they also are training to accuracy in the expected amount of time so it's not just about putting the model out there and open sourcing it and saying here go with it it's about making sure it actually works as the community would expect it to work again some numbers because what good is it if I don't show any numbers in the performance talk so one of the numbers here I want to call out on this slide is just that for cloud TP use the cost to Train image not to 76 percent accuracy is under $85 and we think it's really exciting to be able to just see what you can achieve by making use of this platform I also want to call it if you want more numbers than what's shown here you can definitely take a look at the dawn bench entry that cloud TP you submitted for the image net training one final exciting thing I want to call out about cloud GPUs is the availability of pods which will be coming later this year so what's a pod a pod is actually sixty-four of the devices like I showed earlier all wired up together and you get about 11.5 petaflop of computation in a pod so that's a lot of compute power that is going to be available in this and what can you do with that compute power one of the things that the team has been pushing on is training resonant 50 on a cloud TPU pod to accuracy in less than 15 minutes so we're very excited just to be able to see what can be done with this type of hardware and just the amazing speed that you can get that wasn't really possible before so regice talked about the api's and just the ease of use and things we're focusing on I've given you some fun numbers but what happens when you put it together what content SAR flow actually do so I want to invite Jeff Dean who is the leader of the brain team to come up and talk a bit more about how tensorflow addresses real problems Thanks thanks Megan so I think one of the really remarkable things about machine learning is its capacity to solve real problems in the world and we've seen tremendous progress in the last two years in 2008 though the US National Academy of Engineering put out this list of grand engineering challenges that they were hoping would be solved by the end of the 21st century it's 14 different challenges and I think it's a really nice list of things we should be aspiring to work on as a society you know if we solved all these problems our planet would be healthier people would live longer they would be happier and generally things would be better I think machine learning actually is gonna help us in all of these things some in small ways where you know you see machine learning influencing our understanding of chemical molecules and so on some in very major ways so I'm going to talk about two of these today but I think machine learning will actually be a key to helping us attack all of these different areas the two I'm gonna talk about our engineering the tools for scientific discovery and advancing health informatics so first engineering the tools for scientific discovery you know clearly machine learning is going to be a big component of what we do to solve some of these challenges and tensorflow itself you can think of as a tool for helping us engineer some of these discoveries but one of the things that I think is really important is that there's a lot more opportunity for machine learning then there is machine learning expertise in the world the way you solve a machine learning problem today is you have some data you have some computation maybe GPUs or TP use or CPUs or whatever and then you have a machine learning expert someone who's taking a maybe a graduate class and machine learning someone who's downloaded tensorflow and is familiar enough to play around with it but that's a pretty small set of people in the world and then you stir all this together and you get a solution hopefully so that's the unfortunate thing about that is there's probably tens of thousands of organizations in the world today that are actually effectively using machine learning in production environments and really making use of it to solve problems but there's probably 10 million tens of millions organizations in the world that have data in an electronic form they could be used for machine learning but don't really have the internal expertise and skills so one of the things we're excited about is how can we make machine learning much easier to use so that you don't need nearly as much expertise to apply it so can we use computation to replace a lot of the need for that machine learning expertise so we've been working on a technique a suite of techniques that we call auto auto ml and neural architecture search is one example of this one of the things that machine learning expert does is they sit down and they decide for a particular problem how they what kind of model structure they're gonna use for this problem is it going to be a you know a resonant 50 architecture or is it going to be a nine layer CNN with these kinds of filter sizes and so on it turns out that you can use machine learning to optimize a controller that proposes machine learning models and so you can actually have the controller propose machine learning models train those models on the problem you care about see which one's work well and which ones don't and then use that feedback signal as a reinforcement learning signal for the model generating model it can sort of steer it towards models that are working well for particular kinds of problems and away from the part of the space where those models don't seem to really work very well and if you repeat this a lot of times you can actually get really really powerful high quality models and they look a little weird so this is not something a human machine learning expert would sit down and sort of construct but it has characteristics of things we know that human machine learning experts have discovered are helpful so if you think of the resonant architecture it has these skipped connections that allow you to skip every other layer and so these sort of more organic looking kinds of connections are the same fundamental idea which is you want to allow input data to flow more directly to the output without necessarily going through as much computational layers so the interesting thing is Auto ml actually does quite well here so every dot here this is a graph showing computational cost versus accuracy for imagenet and every dot here shows you different kinds of trade-offs there and generally as you extend more computation you get higher accuracy but every dot here is sort of the work of years of effort a cumulative effort by top machine researchers and computer vision experts in the world and if you look at Auto ml if you run Auto ml you actually get better accuracy and better computational trade-offs than all of those kinds of models that's true both at the high end where you care about our most accuracy and don't care that much about computational budget but it's also true at the low end where you have very lightweight models where you care about look very low computational cost and high accuracy so that's pretty exciting I think we this is a real opportunity to use more computation to solve real machine learning problems in a much more automated way so that we can we can solve more problems more quickly we've actually released this in collaboration with the cloud group at Google as they as a product that customers can use for solving their own computer vision problems and obviously we want to broaden it out from beyond just computer vision to lots of other kinds of categories of problems okay advanced help them form attics so machine learning and healthcare is going to be a really impactful combination one of the areas that we've been working on is a variety of different medical imaging problems including one problem in ophthalmology where you're trying to look at an image like this and diagnose whether that image shows signs of diabetic retinopathy this is a serious degenerative eye disease 400 million people are at risk of this around the world and it's very treatable if it's caught in time but if it's not caught in time you can actually suffer vision loss so and they're often in many parts of the world there just aren't enough ophthalmologists to inspect these images and so we've done work and with work that we published in the very end of 2016 we showed that we had a model that was on par with board-certified ophthalmologists and since then we've been continuing to work on this and we've changed how do we sort of label our training data you've gotten retinal specialist to label the the training data rather than general ophthalmologists and then we actually have a model that is on par with retinal specialists this is a much higher standard of care and we're pretty excited about this because this means you can bring this and deliver this to lots and lots of places around the world but more interestingly I'm going to tell you a little tale of scientific discovery so we had a new person join the the retinopathy team and as a warm-up exercise Lily panghu leads some of this work said that this new person hey why don't you go off and then see if you can predict age and gender from these images and she thought maybe you can predict age within a couple of decades and you definitely shouldn't be able to preach gender so your AEC should be 0.5 so the person went off and came back a little while later and they said I can predict gender with a you see 0.7 and Lily was like mmm that's weird that must be bro lunch go off and come back later and just double check of it in so he come back and they said my AUC is now 0.85 and so that got got us thinking and we investigated what other kinds of things we could predict from these retinal images and it turns out that you can predict a variety of different things that are indicated of cardiovascular health you know your age and gender our symptoms are signs of cardiovascular health as are things like your hemoglobin level lots of things like this we now actually have a new non-invasive test for cardiovascular health normally you'd have to draw blood and do lab tests for this but now we can actually do this just from an image which is pretty cool we're also doing a bunch of work on predictive tasks for healthcare so given a patient's medical record can we predict the future this is something doctors want to do you want to understand how your patients going to progress and you want to be able to answer lots of kinds of questions will the patient be readmitted if I release them from the hospital now what are the most likely diagnosis I should be thinking about what tests should be considered for the station right now lots of questions like that and we've been collaborating with several healthcare organizations to work on de-identified medical records to see if we can predict these kinds of things and in January we posted a many author archives paper and we we looked at a lot of these different kinds of tasks I'll highlight just one of them here which is predicting which patients are most at risk for mortality and using this we're able to actually predict which patients are most seriously at risk 24 hours earlier than the clinical baselines that are currently in use and so that really means the doctors get 24 hours of advance notice to really pay attention to those patients that are that are critically ill and really need their close attention and close I was watching this is indicative of what machine learning can do for this and if you yelled in general the Google brain teams mission is to make machines intelligent and then use that ability to improve people's lives I think these are good examples of where there's real opportunity for this I'm going to close with a bit of a story so when I was five years old I lived in north western Uganda for a year and the local crop there is a route called cassava and I was five so I like to go out and help help people picks cassava but it turns out that machine learning and cassava have a kind of cool twist together so here please roll the video cassava is a really important crop it provides for over 500 million Africans every day when all other crops fail farmers know that they could rely on their cassava plants to provide them food not going to go any home there are several diseases that affect cassava and these diseases makes the fruit an edible it is very crucial to actually control and manage these diseases so we're trying to use machine learning to respond to those diseases and tensorflow is the best foundation for our solutions the apps that we've designed can diagnose multiple diseases it's called narew it's what he leave for life the light that farmers can use to see their problems and find solutions you wave your phone over at a specific leaf look at it and if it has a symptom but box will pop up saying you have this problem when you get a diagnosis we have an option for you to get advice and learn about the best management practices the object detection that we use to tensorflow relies upon our team annotating images we've collected over 5,000 high quality images of different cassavetes diseases for this project we use a single shot detector model on a mobile net architecture it's able to make predictions in less than one second and having to implement thousands of lines of code tensorflow provides a library of functions that allow us to build architectures in much less time we need something that can be deployed on a phone without any connection tensorflow is able to shrink these Ural networks to be able to fit on your mobile device the human input is absolutely critical we're really building something that augments your experience and makes you better at your job so with AI tools and machine learning you can improve the yields you can protect your crops and you can have a much more reliable source of food ai offers the prospect to fundamentally transform the life of hundreds and millions of farmers around the world when LM abundantly onna when you love somebody you can see a product that can actually make someone's life better this is kinda revolution [Music] pretty cool and I think we have some members of the Penn State and IIT a teams from Tanzania here today so if you could all stand up or waiver and I'm sure they'd be happy to chat with you fantastic I'm sure they'll be happy to chat with you at the break about that work so [Music] [Music]