TensorFlow World 2019 Keynote

Key Takeaways

I'm really excited to be here I think it was almost four years ago to the day that we were about 20 people sitting in a small conference room and one of the Google buildings we've woken up early because we wanted a kind of time this for an early East Coast launch where we were turning on the tent of flow org website and releasing the first version of tensorflow as an open source product at project and I'm really really excited to see what it's become it's just remarkable to see the growth and all the different kinds of ways in which people have used this system for all kinds of interesting things around the world so one thing that's interesting is the growth in the use of tensor flow also kind of mirrors the growth in interest in machine learning and machine learning research generally around the world so this is a graph showing the number of machine learning archived papers that have been posted over the last sort of 10 years or so and you can see it's growing quite quite rapidly much more quickly than you might expect and but that lower red line is kind of the nice doubling every couple of years growth rate exponential growth rate we got used to in computing power for due to Moore's law for so many years that's now kind of slowed down but you can see that the machine learning research community is generating research ideas at faster than that right which is pretty remarkable we've replaced computational growth with growth of ideas and we'll see those both together will be important and really the excitement about machine learning is because we can now do things we couldn't do before right as little as five or six years ago computers really couldn't see that well and starting in about you know 2012 2013 we started to have people use deep neural networks to try to tackle computer vision problems image classification object detection and things like that and so now using deep learning and deep neural networks you can feed in the raw pixels of an image and fairly reliably get a prediction of what kind of object is in that image you know feeding the pixels they're red green and blue values in a bunch of different coordinates and you get out the prediction Leopard this works for speed too so while you can feed in audio waveforms and by training on lots of audio waveforms and transcripts of what's being said in those waveforms we can actually take a completely new recording and tell you what is being said I made a transcript well Jerome Italia you can even combine these ideas and have models that take in pixels and instead of just predicting classification classifications of water is in the object can actually write a short sentence a short caption that a human might write about about the image you know a cheetah lying on top of a car that's one of my vacation photos which was kind of cool and so just to show the progress in computer vision in 2011 Stanford hosts an image net contest every year to see how well compute computer vision systems can predict one of a thousand categories in a full-color image and you get about a million images to train on and then you get you know a bunch of test images you've your model has never seen before and you make it need to make a prediction in 2011 the winning entrant got twenty six percent error right so you can kind of make out what that is but it's pretty hard to tell we know from a human experiment that human error of a well-trained human someone who's practiced at this particular task and really understands the thousand categories gets about five percent error this is not a trivial task and in 2016 the winning entrant got three percent error so just look at that tremendous progress in the ability of computers to resolve and understand computer imagery and and and have computer vision that actually works this is remarkably important in the world because now we have systems that can perceive the world around this and can we can do all kinds of really interesting things that we've seen similar progress in speech recognition and language translation and things like that so for the rest of the talk I'd like to kind of structure it around this nice list of 14 challenges that the US National Academy engineering put out and felt like these were important things for the science and engineering communities to work on for the next hundred years they they put this out in 2008 and came up with this list of 14 things after some deliberation and I think you'll agree that these are sort of pretty pretty good large challenging problems that if we actually make progress on them till we'll actually have you know you know a lot of progress in the world we'll be healthier we'll be able to learn things better we'll be able to develop better medicines you know we'll have all kinds of interesting energy solutions so I'm going to talk about a few of these and the first one I'll talk about is restoring and improving urban infrastructure so we're on the cusp of the sort of widespread commercialization of a really interesting new technology that's going to really change how we think about transportation and that is autonomous vehicles and you know this is a problem that has been worked on for quite a while but it's now starting to look like it's actually completely possible and commercially viable to produce these things and a lot of the reason is that we now have computer computer vision and machine learning techniques that can take in sort of raw forms of data that the sensors on these cars collect you know so they have like the spinning light R's on the top that give them 3d point cloud data they have cameras and lots of different directions they have radar in you know the front bumper and the rear bumper and they can really take all this raw information in and with a deep neural network fuse it all together to build a high level understanding of what is going on around the car or is it like at another car door my side there's a pedestrian up here to the left there's a light post over there I don't really need to worry about that moving and really help to understand the environment in which they're operating and then what actions can they take in the world that are both legal safe obey all the traffic laws and get them from A to B and this is not some distant far-off dream alphabets way mo subsidiaries actually been running tests in Phoenix Arizona normally when they run tests they have a safety driver on the front seat ready to take over if the car does something kind of unexpected but for the last year or so they've been running tests in Phoenix with real passengers in the back seat and no safety drivers in the front seat running around suburban Phoenix so suburban Phoenix is a slightly easier training ground than say downtown Manhattan or San Francisco but it's still something that is like not really far off it's something that's actually happening and this is really possible because of things like machine learning and the use of tensorflow in in these systems another one that I'm really really excited about is advanced health informatics this is a really broad area and I think there's lots and lots of ways that machine learning and the use of healthcare data can be used to make better healthcare decisions for people so I'll talk about one of them and really I think the potential here is that we can use machine learning to bring the wisdom of experts through a machine learning model anywhere in the world and that's really a huge huge opportunity so let's look at this through one problem we've been working on for a while which is diabetic retinopathy so diabetic retinopathy is the fastest growing cause of preventable blindness in the world and screening every year if you're at risk for this and if you're if you have diabetes or early sort of symptoms that make it likely you might develop diabetes you should really get screened every year so there's 400 million people around the world that should be screened every year but the screening is really specialized doctors can't do it you really need a ophthalmologist level of training in order to do this effectively and the impact of this shortage is significant so in India for example there's a shortage of a hundred and twenty seven thousand eye doctors to do this sort of screening and as a result 45 percent of patients who are diagnosed to this disease actually have suffered either full or partial vision loss before they're actually diagnosed and then treated and this is completely tragic because this disease if you catch it in time is completely treatable there's a very simple 99% effective treatment that we just need to make sure that the right people get treated at the right time so what can you do so it turns out diabetic retinopathy screening is also a computer vision problem and the progress we've made on general computer vision problems where you want to take a take a picture and tell if that's a leopard or an aircraft or a car actually also works for diabetic retinopathy so you can take a retinal image is what the screaming camera sort of the raw data that comes off the screening camera and try to feed that into a model that predicts one two three four or five that's how these things are graded you know one being no diabetic retinopathy five being proliferative and the other numbers being in between so turns out you can get a collection of data of retinal images and have ophthalmologists label them turns out if you ask two ophthalmologists to label the same image they agree with each other sixty percent of the time on the number one two three four or five but perhaps slightly scarier if you ask the same ophthalmologist degrade the same image a few hours apart they agree with themselves sixty five percent of the time but you can fix this by actually getting each image labeled by a lot of ophthalmologists so you get it labeled by seven ophthalmologists if five of them say at the - and two of them say it's a three it's probably more like a tooth and a three eventually you have a nice high quality it is that you can train on like many machine learning problems high quality data is the right raw ingredient but then you can apply basically an off-the-shelf computer vision model trained on this data set and now you can get a model that is on par or perhaps slightly better than the average board-certified ophthalmologist Sameer s which is pretty amazing it turns out you can actually do better than that and if you get the data labeled by retinal specialists people who have more training in retinal disease and have and changed the protocol by which you label things you get three retinal specialists to look at an image discuss it amongst themselves and come up with a what's called a I sort of coordinated assessment and one one number then you can train a model and now be on par with retinal specialist which is kind of the gold standard of care in this area and that's something you can now take and distribute widely around the world so one issue with with particularly with healthcare kinds of problems is you want explainable models you want to be able to explain to a clinician you know why is this person why do we think this person has moderate diabetic retinopathy so you can take a retinal image like this and one of the things that really helps is if you can show in the models assessment why this is a 2 and not a 3 and by highlighting parts of the input data you can actually make this more understandable for clinicians and enable them to sort of really sort of get behind the assessment that the model is making and we've seen this in other areas as well it's been a lot of work unexplained ability so I think the notion that deep neural networks are sort of complete black boxes it's a bit overdone there's actually a bunch of good techniques that are being developed and more all the time that will improve this so a bunch of advances depend on being able to understand text so and we've had a lot of really good improvements in the last few years on language understanding so this is a bit of a story of research and how research builds on other research so in 2017 a collection of Google researchers and interns came up with a new kind of model for text called the transformer model so unlike recurrent models where you have kind of a sequential process where you absorb one word or one token at a time and update some internal stage and then go on to the next token the transformer model enables you to process a whole bunch of text all at once in parallel making it much more computationally efficient and then to use attention on previous text to really focus on if I'm trying to predict what the next word is you know what are other parts of the context to the left that are relevant to predicting that so that paper was was quite successful and showed really good results on language translation tasks with a lot less compute so the blue score there and the first two columns for English to German and English to French higher is better and then the the compute cost of these models shows that this is getting sort of state-of-the-art results at that time with 10 to 100 X less compute than other approaches then in 2018 another team of Google researchers built on the idea of transformers so everything you see there in a blue oval is a transformer module and they came up with this approach is called bi-directional encoder encoding representations from transformers or Burtt we it's a little bit shorter and more catchy so Burt has this really nice property that in addition to using context to the left it uses context all around the language the the sort of the surrounding text in order to make predictions about text and the way it works is you start with a self supervisor objective so the one really nice thing about this is there's lots and lots of text in the world so if you can figure out a way to use that text to train a model to be able to understand text better that would be great so we're gonna take this text and in the bird training objective to make it self supervised we're gonna drop about 15 percent of the words and this is actually pretty hard but the model is then gonna try to fill in the blanks essentially try to predict what are the missing words that were dropped and because we actually have the original words we now know you know if the model is correct and it's guesses about what goes in the box and by processing trillions of words or texts like this you actually get a very good understanding of contextual cues in language and how to actually fill in the blanks in a really intelligent way and so that's essentially the training objective for bird you take text you drop 15 percent of it and then you try to predict those missing words and one key thing that works really well is that's step one you can pre train a model on lots and lots of text using this fill in the blanks self supervisor objective function and then step two you can then take a language task you really care about like maybe you want to predict is this a you know a five star review or a one star review for some hotel but you don't have very much labeled text for that for that actual task you might have 10,000 reviews and know the star count of each review but you can then fine-tune the model starting with the model trained in step one on trillions of words of text and now use your paltry 10,000 examples for the text task you really care about and that works extremely well so in particular Bert gave state-of-the-art results across a broad range of different text understanding benchmarks in this glue benchmark suite which was pretty cool and people have been using now in this way to improve all kinds of different things all across the language understanding in an LP space so one of the grand challenges was engineer the tools of scientific discovery and I think it's pretty clear machine learning is actually going to be an important component of making advances in a lot of these other Grand Challenge areas things like autonomous vehicles or other kinds of things and it's been really satisfying to see what we'd hoped would happen when we release tensorflow as an open source project has actually kind of come to come to pass as we were hoping in that lots of people would sort of pick up tensorflow use it for all kinds of things people would improve the core system they would use it for tasks we would never imagine and that's been quite satisfying so people have done all kinds of things some of these are our uses intra inside of Google some are outside inside academic institutions some are you know scientists working on conserving whales or understanding like ancient scripts many kinds of things which is pretty neat the breadth of breadth of uses is really amazing this these are the 20 winners of the google.org a I impact challenge where people could submit proposal for how they might use machine learning and AI to really tackle a local challenge they saw in their communities and they have all kinds of things keep ranging from like trying to predict better ambulance dispatching to identifying sort of illegal logging using speech recognition or audio processing pretty neat and many of them are using tensorflow so one of the things we're pretty excited about is auto ml which is this idea of automating some of the process by which machine learning experts sit down and sort of make decisions to solve machine learning problems so currently you have a machine learning expert sit down they take data they have computation they run a bunch of experiments they kind of stir it all together and eventually you get a solution to a problem you actually care about one of the things we'd like to be able to do though is see if we could eliminate a lot of a need for the human machine learning expert to run these experiments and instead automate the experimental process by which a machine learning expert comes by a high-quality solution for a problem you care about so one of the you know lots and lots of organizations around the world have machine learning problems but many many of them don't even realize they have a machine learning problem let alone have people in their organization that can tackle the problem so one of the earliest pieces of work our researchers did in the space was something called neural architecture search so when you sit down and design a neural network to tackle a particular task you make a lot of decisions about you know shapes of this and that and like should it be used three by three filters in layer 17 or 5x5 all kinds of things like this it turns out you can automate this process by having a model generating model and train the model generating model based on feedback about how well the models that it generates work on the problem you care so the way this will work we're gonna generate a bunch of models those are just descriptions of different neural network architectures we're gonna train each of those for a few hours and then we're going to see how well they work and then use the accuracy of those models as a reinforcement learning signal for the model generating model to steer it away from models that didn't work very well and towards models that worked better and we're gonna repeat many many times and over time we're gonna get better and better by steering the search to the parts of the space of models that worked well and so it comes up with models that look a little strange admittedly you know human probably would not sit down and wire up a sort of machine learning computer vision model exactly that way but they're pretty effective so if you look at this graph this shows kind of the best machine human machine learning experts computer vision experts machine learning researchers in the world producing a whole bunch of different kinds of models of in the last four or five years things like ResNet 50 dense net 201 inception ResNet all kinds of things that black dotted line is kind of the frontier of human machine learning expert model quality on the y-axis and computational cost on the x-axis so what you see is as you the x-axis you tend to get more accuracy because you're applying more computational cost but what you see is the blue dotted line is Auto ml based solutions systems where we've done this automated experimentation instead of pre designing any particular architecture and you see that it's better both at the high end where you care about most accurate model you can get regardless of computational cost but it's also accurate at the low end where you care about a really lightweight model that might run in a phone or something like that and in 2019 we've actually been able to improve that significantly this is a set of models called efficient net and has a very kind of a slider about you can trade-off computational cost and accuracy but they're all way better than human sort of guided experimentation on the black that black dotted line there and this is true for image recognition for classification it's true for object detection so the red line there is auto ml the other things are not it's true for language translations so the black line there is various kinds of transformers the red line is we've gave the basic components of transformers to an auto ml system and allowed it to fiddle with it and come up with something better it's true for computer vision models used in autonomous vehicles so this is a collaboration between way moe and google research we were able to come up with models that are you know significantly lower latency for the same quality or we could trade it off and get significantly lower error rate at the same latency it actually works for tabular data so if you have lots of like customer records and you want to predict which customers or you know gonna be spending a thousand dollars with your your business next month you know you can use auto ml to come up with a high quality model for that kind of problem okay so what do we want I think we want the following properties in that computer on a machine learning model so one is we tend to Train separate models for each different problem we care about I think this is a bit misguided like really we want one model that does a lot of things so that it can build on the knowledge in how it does thousands or millions of different things so that when the million and first thing comes along it can actually use its expertise from all the other knows how to do to know how to get into a good state for the new problem with relatively little data and relatively little computational cost so these are some nice properties I have kind of a cartoon diagram of something I think might make sense so imagine we have a model like this where it's very sparsely activated so different pieces of the model you know I'll have different kinds of expertise and they're called upon when it makes sense but they're mostly idle so it's relatively computational even power efficient but it can do many things and now each component here is some piece of a machine learning model with different kinds of state parameters in the model and different operations and a new task comes along now you can imagine something like neural architecture search becoming squint at it just right and now turn it into a neural pathway search we're gonna look for components that are really good for this new task we care about and maybe we will search and find that this path through the model actually gets us into a pretty good state for this new task because maybe it goes through components that are trained on related tasks already and now maybe we want that model to be more accurate for the purple tasks so we can add a bit more you know computational capacity add a new component start to use that component for this new task continue training it and now that new component can also be used for solving other related tasks and each component itself might be running some sort of interesting architectural search inside it so I think something like that is the direction we should be exploring as a community it's not what we're doing today but I think it could be a pretty interesting direction okay and finally I'd like to touch on thoughtful use of AI in societies we've seen more and more uses of machine learning in our products and around the world it's really really important to be thinking carefully about how we want to apply these technologies you know they can like any technology these systems can be used for amazing things or things we might find a little sort of detrimental in various ways and so we've come up with a set of principles by which we think about applying sort of machine learning and AI to our products and we've made these public about a year and a half ago a way of sort of sharing our thought process with the rest of the world and I pretty particularly like these I'll point out many of these are sort of areas of research that are not fully understood yet but we aim to apply the best in the state-of-the-art methods for example for reducing bias in machine learning models but also continue to do research and advance the state of the Artemis areas and so this is just kind of a taste of different kinds of work we're doing in this area how do we do machine learning with more privacy using things like federated learning how do we make models more interpretive also the clinician can understand the predictions is making on on diabetic retinopathy sort of examples how do we make machine learning more fair okay and with that I hope I've convinced you that deep neural Nets and machine learning you're already here so maybe you're already convinced to this but are helping make sort of significant advances and a lot of hard computer science problems computer vision speech recognition language understanding general use of machine learning is going to push the world forward so thank you very much and I appreciate you all being here hey everyone good morning I just want to say first of all welcome today I want to talk a little bit about tensorflow 2.0 and some of the new updates that we have that are gonna make your experience with tensor flow even better but before I dive into a lot of those details I want to start off by thanking you everyone here everyone on the livestream everyone who's been contributing to tensorflow all of you who make up the community tensorflow was open source to help accelerate the AI field for everyone you've used it in your experiments you've deployed it in your businesses you've made some amazing different applications that were so excited to showcase and talked about some that we get to see a bit here today which is one of my favorite parts about conferences like this and you've done so much more and all of this has helped make tensorflow what it is today it's the most popular ml ecosystem in the world and honestly that would not happen without the community being excited and embracing and using this and giving back so on behalf of the entire tensorflow team I really just first want to say thank you it's so amazing to see how tensorflow is used that's one of the greatest things I get to see about my job is the applications and the way folks or using tensorflow I want to take a step back and talk a little bit about some of the different user groups and how we see them making use of tensor flow tensor flow is being used across a wide range of experiments and applications so here calling out researchers data scientists and developers and there's other groups kind of in between as well researchers use it because it's flexible it's flexible enough to experiment with and push the state of the art and deep learning you heard this even just a few minutes ago with folks from Twitter talking about how they're able to use tensor flow and expand on top of it in order to do some of the amazing things that they want to make use of on their own platform and at Google we see examples of this when researchers are creating advanced models like excel nap and some of the other things that Jeff referenced in his talk earlier taking the step forward looking at data scientists data scientists and enterprise engineers have said they rely on tensorflow for performance and scale in training and production environments that's one of the big things about tensorflow that we've always emphasized and looked at from the beginning how can we make sure this can scale to large production use cases for example quantify in Blackrock use tensor flow to test and deploy Bert in real-world NLP instances such as text tokenization as well as classification hopping one step forward looking a bit at application developers application developers use tensorflow because it's easy to learn ml on the platforms that they care about Arduino wants to make ml simple on microcontrollers so they rely on tensorflow pre-trained models and tensorflow light micro for deployment each of these groups is a critical part of the tensorflow ecosystem and this is why we really wanted to make sure that tensorflow tirado works for everyone we announced the Alpha at our dev summit earlier this year and over the past few months the team has been working very hard to incorporate early feedback again thank you to the community for giving us that early feedback so we can make sure we're developing something that works well for you and we've been working to resolve bugs and issues and things like that and just last month in September we were excited to announce the final general release for tensorflow to dotto you might be familiar with Tenzer flows architecture which has always supported the ml lifecycle from training through deployment again one of the things we've emphasized since the beginning when tensorflow was initially open sourced a few years ago but I want to emphasize how tends to flock to dotto makes this workflow even easier and more intuitive first we invested in Karis and using an easy-to-use package and tensorflow making it the default high level API many developers love Karis because it's easy to use and understand again you heard this already mentioned a little bit earlier and hopefully we'll hear more about it throughout the next few days by tightly integrating Karis into 2.0 we can make Karis work even better with primitives like TF data we can do performance optimizations behind the scenes and run distributed training again we really wanted to tato to focus on usability how can we make it easier for developers how can we make it easier for users to get what they need out of tensorflow for instance lose it a customized weight loss app said they use TF Kerris for designing their network by leveraging mirrored strategy distribution into dot o they were able to utilize the full power of their GPUs it's feedback like this that we love to hear and again it's very important for us to know how the community is making use of things how the community is using to do the things they want to see so that we can make sure we're developing the right framework and also make sure you can contribute back when you need a bit more control to create advanced algorithms tirado comes fully loaded with eager execution making it familiar for Python developers this is especially useful when you're stepping through doing debugging making sure you can really understand step-by-step what's happening this also means there's less coding required when training your model all without having to use session that run again usability is a focus to demonstrate the power of training models with 2.0 you how you can train a state-of-the-art NLP model in ten lines of code using the Transformers NLP library by hugging face again a community contribution this popular package hosts some of the most advanced NLP models available today like Bert GPT transformer Excel Excel net and now supports tensor flow to dotto so let's take a look here kind of just looking through the code you can see how you can use tirado to Train hugging faces distill burp model for text classification you can see you just simply load the tokenizer model and the data set then prepare the data set and use TF Kerris compile and fit API s-- and with a few lines of code I can now train my model and with just a few more lines we can use the train model for tasks such as text classification using eager execution again it's examples like this where we can see how the community takes something and is able to do something very exciting and amazing by making use of the platform and the ecosystem the tensorflow is providing but building and training a model is only one part of tensorflow 2.0 you need the performance to match that's why we worked hard to continue to prove performance with tensorflow to dato it delivers up to 3x faster training performance using mixed precision on nvidia Volta and touring GPUs in a few lines of code with models like resonant 50 and Bert as we continue to double down on 2.0 in the future performance will remain a focus with more models and with hardware accelerators for example in 2.1 so the next upcoming tensorflow release you can expect TPU and TPU pods support along with mix precision for GPUs so performance is something that we're keeping a focus on as well while also making sure usability really stands to the forefront but there's a lot more to the ecosystem so beyond model building and performance there are many other pieces that help round out the tensorflow ecosystem add-ons and extensions are a very important piece here which is why we wanted to make sure that they're also compatible with tensorflow - dotto so you can use popular libraries like some other ones called out here whether it's tensorflow probability TF agents or TF text we've also introduced a host of new libraries to help researchers and email practitioners and more useful ways so for example neural structure learning helps to Train neural networks with structured signals and the new fairness indicators add-on enables regular computation and visualization of fairness metrics and these are just the types of things that you can see kind of as part of the tensorflow ecosystem these add-ons that again can help you make sure you're able to do the things you need to do not with your models but kind of beyond just that another valuable aspect of the tensorflow ecosystem is being able to analyze your ml experiments in detail so this is showing tensor board tensor board as tensor flows visualization toolkit which is what helps you accomplish this it's a popular tool among researchers and email practitioners for tracking metrics visualizing model graphs and parameters and much more it's very interesting that we've seen users enjoy tensor boards so much it'll even take screenshots of their experiments and then use those screenshots to be able to share with others what they're doing with tensor flow this type of sharing and collaboration in the email community is something we really want to encourage with tensorflow again there's so much that can happen by enabling the community to do good things that's why I'm excited to share the preview of tents aboard dev a new free managed tensor board experience that lets you upload and share your ml experiment results with anyone you'll now be able to host and track your ml experiments and share them publicly no setup required simply upload your logs and then share the URL so that others can see the experiments and see the things that you are doing with tensorflow as a preview we're starting off with a scalers dashboard but over time we'll be adding a lot more functionality to make the sharing experience even better but if you're not looking to build models from scratch and want to reduce some computational cost tensorflow has always made pre-trained models available through tensorflow hub and today we're excited to share an improved experience of tensorflow hub that's much more intuitive where you can find a comprehensive repository of pre trained models in the tensor flow ecosystem this means you can find models like Bert and others related to image text video and more that are ready to use with tensorflow light and tensorflow j/s again we wanted to make sure the experience here is vastly improved to make it easier for you to find what you need in order to more quickly get to the task at hand tensorflow is driven by all of you tensorflow hub is hosting more treat pre train models from the community you'll be able to find curated models by deep mine Google Microsoft's AI for Earth and NVIDIA ready to use today with many more to come we want to make sure that tensorflow hub is a great place to find some of these excellent pre train models and again there's so much the community is doing we want to be able to showcase those models as well tensorflow 2.0 also highlights tensor flows core strengths and areas of focus which is being able to go from model building experimentation through to production no matter what platform you work on you can deploy into in ml pipelines but tensorflow extended or tf-x you can use your models on mobile and embedded devices with tensor flow light for on device inference and you can train and run models in the browser or nodejs with tensorflow ojs you'll learn more about what's new in tensor flow in production during the keynote sessions tomorrow you can learn more about these updates by going to tensorflow org where you'll also find the latest documentation examples and tutorials for Toronto again we want to make sure it's easy for the community to see what's happening what's new and enable you to just do what you need to do with tensorflow we've been thrilled to see the positive response to do out to 2.0 and we hope you continue to share your feedback thank you and I hope you enjoy the rest of TF world hello everyone I'm Fred Ryce I work for IBM I've been working for IBM since 20 2006 and I've been contributing to tensorflow course since 2017 but my primary job at IBM is to serve as tech lead for code 8 that's the Center for open source data and AI technologies we are an open source lab located in downtown San Francisco and we work on open source technologies that are foundational to AI and we have on staff 44 full-time developers who work only on open source software now that's that's a lot of developers a lot of open source developers or is it well if you look across IBM at all of the IBM Burres who are active contributors to open source in that they have committed code to github in the last 30 days you'll find that there are almost 1,200 IBM errs in that category so our 44 developers are actually a very small slice of a very large pie oh and those numbers they don't include Red Hat when we close that acquisition earlier this year we more than doubled our number of active contributors to open source so you can see that iBM is really big in open source and more and more the bulk of our contributions in open in the open are going towards the foundations of AI and when I say AI I mean AI in production I mean AI at scale AI at scale is not an algorithm it's not a tool it's a process it's a process that starts with data and then that data turns into features and those features train models and those models get deployed in applications and those applications produce more data and the whole thing starts all over again and at the core of this process is an ecosystem of open source software and at the core of this ecosystem is tensorflow which is why I'm here on behalf of IBM open source to welcome you to tensor for the world now throughout this conference you're going to see talks that speak to all of the difference stages of this AI lifecycle but I think you're going to see a special emphasis on this part moving models into production and one of the most important aspects of moving models into production is that when your model gets deployed in a real world application it's going to start having effects on the real world and it becomes important to ensure that those effects are positive and that they're fair to your clients to your users now at IBM here's a hypothetical example that our researchers put together about a little over a year ago they've took some real medical records data and they produced a model that predicts which patients are more likely to get sick and therefore should get additional as screening and they showed that if you naively train this model you end up with a model that has significant racial bias but that by deploying state-of-the-art techniques to adjust the data set and the process of making the model they could substantially reduce this model to produce a model reduce this bias to produce a model that is much less much more fair you can see a jupiter notebook with the entire scenario from end to end including code and equations and results at the URL down here again I need to emphasize this was a hypothetical example we we built a flawed model deliberately so we could show how to make it better but no patients were harmed in this exercise however last Friday I sat down with my morning coffee and I opened up the Wall Street Journal and I saw this article at the bottom of page three describing a scenario eerily similar to our hypothetical you know when your hypothetical starts showing up as new papers headlines that's kind of scary and I think it is incumbent upon us as an industry to move forward the process the the technology of trust in AI trust and transparency in AI which is why IBM and IBM Research have released our toolkits of state-of-the-art algorithms in this space as open source under AI fairness 360 a I explained ability 360 an adversary over a bust mystery six it is also why IBM is working with other members of the Linux Foundation a I trusted a I committee to move forward open standards in this area so that we can all move quickly more quickly to trusted AI now if you'd like to hear more on this topic my colleague Animesh singh will be giving a talk this afternoon at 140 on trusted AI for the full 40 minute session also I'd like to give a quick shout out to my other cook co-workers from code 8 who have come down here to show you cool open-source demos at the IBM booth that's booth 201 I also check out our website's developer ibm.com and co.org on behalf of IBM I'd like to welcome you all to tensorflow world enjoy the conference thank you hi I'm Ted some Iran from Twitter before I get started my conversation today I want to do a quick plug for Twitter what's great about events like this is you get to hear people like Jeff Deen talk and you also get to hear from colleagues and people in the industry that are facing similar challenges is you have conversations around developments and data science and machine learning but what's great is that's actually available every day on Twitter Twitter's phenomenal for conversation on data science and machine learning people like Jeff Dean and other thought leaders are constantly sharing their thoughts and their developments and you can follow that conversation and engage in it and only that but you can bring that conversation back to your workplace and come off looking like a hero just something to consider so without that shame shameless plug my name's Ted Sammy I lead product for cortex cortex is Twitter central machine learning organization if you have any questions for me or the team feel free to connect with me on Twitter we can follow up later so before we get into how we're accelerating ml at Twitter let's talk a little bit about how are you been using ml at Twitter Twitter is largely organized against three customer needs the first of which is our health initiative that might be a little bit confusing to you you might think of it as user safety but we think about it as improving the health of conversations on Twitter and machine learning is already at use here we use it to detect spam we can algorithmically you're not scale to tech spam and protect our users from it similarly in the abuse space we can proactively flag content as potentially abuse toss it up for human reviews and act on it before our users even get impacted by it a third space where we're using machine owning here is something called NSFW not-safe-for-work I think you're all familiar with the acronym so how can we at scale identify this content and handle it accordingly another use of machine learning in this space there's more that we want to do here and there's more that we're already doing similarly the consumer organization this is largely what you think of the big blue app of Twitter and here the customer job whether we're serving is helping connect our customers with the conversations on Twitter that they're interested in and one of the primary veins that we do which we do this is our time line our time line today is ranked so if you're not familiar users follow accounts content and tweets associated with those accounts get funneled into a central feed and we rank that based on your past engagement and interest to make sure we bring forth the most development and most relevant conversations for you now there's lots of conversations on Twitter and you're not following everyone and so there's also a job that we have to serve about bringing forth all the conversations that you're not proactively following but are still relevant to you this is surfaced in our recommendations product which uses machine learning to scan the corpus of content on Twitter and identify what conversations be most interesting to you and push it to you in a notification the inverse of that is when you cuz you know what the topics you want to explore are but you're looking for the conversations around that that's where we use Twitter search this is another surface area and the big blue app that we're using machine learning the third job to be done for our customers is helping connect brands with their customers you might think of this as the ads product and this is actually the og of machine learning at Twitter the first team that implemented it and here we use it for what you might expect ads ranking that's kind of like the timeline ranking but instead of tweets its ads and identifying most relevant ads for our users and as signal is to go into that we also do user targeting to understand your past engagement ads understand which ads or or in your interest space and the third oh yeah are still good and the third is brand safety you might not think about this when you think about machine learning and advertising but if you're a company like United and you wanna advertise on Twitter you want to make sure that your ad never shows up next to a tweet about a plane crash so how do we at scale protect our brands from those off-brand conversations we use machine learning for this as well so as you can tell machine learning is a big part of all of these organizations today and where we have shared interests and shared investment we want to make sure we have a shared organization that serves that and that's the need for cortex central machine-learning team and our purpose is really quite simple to enable Twitter with ethical and advanced AI and to serve that purpose we've organized in three ways the first is our applied research group this group applies the most advanced MLT techniques from industry and research to our most important surface areas whether they be new initiatives or existing places this team you can kind of think of as like an internal task force or consultancy that we can redeploy against the companies top initiatives and his signals when using machine learning having shared data assets that are broadly useful can provide as more leverage examples of this would be our language understanding team that looks at tweets and identifies named entities inside them those can then be offered up as features for other teams to consume in their own applications of machine learning similarly our media understanding team looks at images and can create a fingerprint of any image and therefore we can identify every use of that image across the platform these are examples shared signals that were producing that can be used for machine learning at scale inside the company and the third organization is our platform team and this is really the origins of cortex here we provide tools and infrastructure to accelerate ml development at Twitter we do increase the velocity of our ml practitioners and this is really the focus of the conversation today when we set out to build this ML platform we decided we wanted a shared ml platform across all of Twitter and why is that important that it be shared across all of Twitter well we want transferability we want the great work being done in the ads team to be where possible transferable to the to benefit the health initiative where that's relevant and similarly if we have great talent in the consumer team that's interested in moving to the ads team if they're on the same platform they can transfer without friction be able to ramp up quickly so we set out with this goal of having a shared ml platform across all of Twitter and when we did that we looked at a couple of product requirements first it needs to be scalable it needs to be able to operate a Twitter scale the second it needs to be adaptable this space is developing quickly so we need a platform that can evolve with data science and machine learning developments third is the talent pool we want to make sure that we have a development environment Twitter that appeals to the ml researchers and engineers that we're hiring and developing fourth is the ecosystem we want to be able to lean on the partners that are developing industry-leading tools so we can focus on technologies that are Twitter specific fourth is documentation that you want to understand that we want to be able to quickly unblock our practitioners as they hit the issues which was inevitable in any platform and finally usability we want to remove friction and frustration from the lives of our team so that they can focus on delivering value for our end customers so considering these product requirements let's see how tensorflow has done against them scalability we validated this by putting tensorflow by way of our implementation we call deep Bird against timeline ranking so every tweet that's ranked in the timeline today runs through tensorflow so we can consider that test validated is adaptability then novel architectures that tends to evoke and support as well as the custom loss functions allows us to react to the latest research and an employee that inside the company an example that we published on this publicly is our use of a split net architecture and adds ranking so tensorflow has been very adaptable for us there it is the talent pool and we think about the talent pool and kind of two types there's the ml engineer and the ml researcher and as a proxy of these audiences we looked at the github data on these and clearly tensorflow has widely adopted amongst ml engineers and similarly the archive community shows strong evidence of light adoption in the academic community on top of this proxy data we've also have anecdotal evidence of the speed of ramp up for our male researchers and ml engineers inside the company you the ecosystem whether it's tensor board TF data validation TF model analysis TF meta story of hub TFT effects pipelines there's a slew of these products out there and they're phenomenal they allow us to focus on developing tools and infrastructure that is specific to Twitter's needs and lead on the great work of others so we're really grateful for this intense fluid is great v being documentation now this is what you would go to when you go to tensorflow and you see that phenomenal documentation as well as great education resources but what you might not appreciate we've come to really appreciate is the value of the user-generated content what Stack Overflow and other platforms can provide in terms of user-generated content is almost as valuable as anything tensorflow itself can create and so tensorflow giving it's a widespread adoption and it's great tensorflow website has provided phenomenal documentation for ml practitioners finally usability and this is why we're really excited about tensorflow 2.0 the orientation around the Charis api makes it more user friendly it also still continues to allow for flexibility for more advanced users the eager execution enables more rapid and intuitive debugging and it closes the gap between mo engineers and modelers so clearly from this checklist we're pretty happy with our engagement with tensorflow we're excited about continuing developed a platform with them and push the limits on what it can do gratitude to the community for their participation and involvement in the product and appreciate their conversation on twitter and as we advance it so if you have any questions for me as I said before you can connect with me but I'm not alone here today a bunch of my colleagues are here as well so if you see them roaming the halls feel free to engage with them or as I shared before you can continue the conversation on Twitter here are their handles thank you for your time [Applause] I just want to begin by saying I've been dabbling in like cloud a I and cloud machine learning for a while and during that time I it never occurred to me that we'd be able to come out with something like we did today because this is only possible because Google Cloud and tensorflow can collaborate unbelievably closely together within within Google so to begin let's let's talk a little bit about tensorflow 46 million downloads tensorflow has been massive growth the last few years it's expanded from from the forefront of research which we've seen earlier this morning to businesses taking it on as a dependency for their business to operate on a day-in day-out basis it's a super exciting piece as someone who spends most of their time most all of their time thinking about how we can bring AI and machine learning into businesses seeing tensor flows commitment and focus on deploying actual ml in production is super exciting to me with this grouse though comes growing pains and part of that is things like support right when my model doesn't do what I expected it to or or my training job fails you know what options do I have and you know how well does your boss respond when you say hey yes I've I don't know why my models not training but not to worry I've put a question on slack and hopefully someone will get back to me you know we understand that businesses who are taking a bet on tensorflow as a critical piece of their hardware architecture or their their stack need more than this second it can be a challenge to unlock the scale and performance of cloud for those of you who like me have gone through this journey over the last couple of years you know for me it started on my laptop right and then eventually I outgrew my laptop and so I had a gaming rig under my desk right with a GPU and eventually there were eight GP there were eight gaming rigs under my desk and when you open the door to my office the whole floor knew because it sounded like Antoine right and but now with today's cloud that doesn't have to be the case you can go from that single instance all the way up to a massive scale you know seamlessly so with that today we bring you tensorflow enterprise tensorflow enterprise is designed to do three things one give you enterprise grade support to cloud scale performance and three managed services when and where you want them at the abstraction level you want them enterprise-grade support what does that mean fundamentally what that means is that as these businesses take a bet on tensorflow many of these business have businesses have IT policies or or requirements that the software have you know a certain longevity before they're willing to commit to it in production and so today for certain versions of tensorflow when used on Google cloud we will extend that one year of sort of support a full three years that means that if you're building models on 11.15 today you can know that for the next three years you'll get bug fixes and security patches when and where you need them simple and scalable scaling from an idea on a single node to production at massive scale can be daunting right you know saying to my to my boss hey I took a sample of the data was something that previously seemed totally reasonable but now we're asked to train on the entire corpus of data and that can take days weeks we can help with all of that by deploying tensorflow on google cloud a network that's been running tensorflow successfully for years and has been highly optimized for this purpose so scalable across our world-class architecture the products are compatibility tested with the cloud their performance optimized for the cloud and for Google's world-class infrastructure what does this mean so if any of you have ever had the opportunity to use bigquery bigquery is Google clouds kind of massively parallel cloud hosted data warehouse and if by the way if you haven't tried using bigquery highly recommend going out and trying it it is it returns results faster than than can be imagined that speed in bigquery we wanted to make sure we were taking full advantage of that and so recent changes and recent pieces included in tensor flow Enterprise have increased the speed of the connection between the data warehouse and and tensor flow by three times right now all of sudden those jobs that were taking days take hours Unity gaming a wonderful customer and partner with us you can see the quote here unity leverages these aspects of tensorflow enterprise in their business their monetization products reach more than three billion devices three billion devices worldwide game developers rely on a mix of scale and products to drive installs and revenue and player engagement and unity needs to be able to quickly test build scale deploy models all at massive scale this allows them to serve up the best results for their developers and their advertisers managed services as I said tensorflow enterprise will be available on Google cloud and will be available as part of Google clouds AI platform will also be available in in vm's if you'd prefer that or in containers if you want to run them on Google Cloud Cooper Nettie's engine or using coop flow on Cooper Nettie's engine in submarine tender tensorflow Enterprise offers enterprise grade support that continuation that full three years of support that IT departments are accustomed to cloud scale performance so that you can run at massive scale and works seamlessly with our managed services and all of this is free in fully included for all Google cloud users Google Cloud becomes the best place to run tensorflow but there's one last piece which is for companies for whom AI is their business not companies for whom AI might help with this part of their business or that or might help optimize this this campaign or this back-end system but for for companies where AI is their business right where they're training hundreds of thousands of hours of training a year petabytes of data right using cutting-edge models to meet their unique requirements we are introducing tensorflow Enterprise with white gloves support this is really for cutting-edge AI right engineering to engineering assistance when needed you know close collaboration across Google allows us to fix bugs faster if needed one of the great opportunities of working in cloud if you ask my kids they'll tell you that the reason I working in cloud a I and and in kind of machine learning is in an effort to keep them ever from learning to drive their eight and ten years old so I need people to kind of hurry along this route if you will but you know one of the partners one of the customers and partners we have is crews automotive and you can see here they're a shining example of the work we're doing on this on their quest towards self-driving cars they've also experienced hiccups and challenges and scaling problems and we've been a critical partner for them in helping ensure that they can achieve the results they need to to solve this kind of generational defining problem of auto of autonomous vehicles you can see not only did we improve the accuracy of their models but also reduce training times from four days down to one day this Cree this allows them to iterate at speeds previously unthinkable so none of this as I said wouldn't would have been possible without the close collaboration between Google Cloud and tensorflow you know I I look back on on Megan's recent announcement of tensor board dev you know we will be looking at bringing that that type of functionality into an enterprise environment as well in the coming months but we're really really excited to get tensorflow Enterprise into your hands today to learn more and get started you can go to the link well as sessions later today so if and if you are on the cutting edge of AI we are accepting applications for the white glove service as well we're excited to bring this offering to teams we're excited to bring this offering to businesses that want to move in into a place where machine learning is increasingly a part of how they create value thank you very much for your time today hi my name is Kemal I'm the project director for tensorflow so earlier you heard from Jeff and Megan about the prog direction now what I'd like to talk about is the most important part of what we're building and that's the community that's you sorry we're just night thank you so as you've seen in the video we've had a great Roadshow 11 events spanning five continents to connect the community with the tensorflow team I personally was very lucky this summer because I got to travel to Morocco and Ghana and Shanghai amongst other places just to meet the community and to listen to your feedback and we heard a lot of great things so as we're thinking about you know how can we best help the community it really came down to three things first we would like you to help you to connect with the larger community and to share the latest and greatest of what you've been building then we also would like you you know we want to help you learn learn about ml learn about tensorflow and then we want to help you contribute and give back to the community so let's start with connect so why connect well first the community the tensorflow community has really grown a lot it's huge it's 46 million downloads 2,100 commuters and again I know that we've been saying that all along but I really want to say a huge thank you on behalf of the tensorflow team for making the community what it is today another aspect of the community that we're very proud of is that it's truly global you know this is a revised map of our of our get up stars and as you can see we're covering all timezone and keep growing so the community is huge it's truly global and I really want to think about how can we bring the community closer together and this is really what initiated the idea of tensorflow world we wanted to create an event for you we want an event where you could come up and connect with the rest of the community and share what you've been working on and this has actually started organically seven months ago the tensorflow user group started and I think now we have close to 50 the largest one is in Korea has 46,000 members we have 15 in China so if you're in the audience or in a live stream and you're looking into this map and you're thinking wait I don't see you know I don't I don't see a dot where I live and you have tensorflow member that you're connecting with and you want to start a tends to flow user group well we'd like to help you so please go to ten-foot org slash community and we'll help you get it started so that next year when we look at this map we have dots all over the place so what about businesses we're talking about developer what about businesses one thing we heard from businesses is you know they have this this business problem they think ml can help them but they're not sure how and that's a huge missed opportunity when we look at the staggering thirteen trillion dollars that AI will bring to the global economy over the next decade so you have those businesses on one side and you have partners on the other side you know who know about ml they know how to use tensorflow so how do we connect those two well this was the inspiration for launching our trusted partner pilot program which helps you as a business connect to a partner we'll help you solve your ml problem so if you go on 10-footer org you'll find more about our trusted partner program just a couple examples of cool things that they've been working on one partner helped a conference company shorten the insurance claim processing time using image processing techniques another partner helped a global med tech company by automating the shipping labeling process using object recognition technique and you'll hear more from this partner later today I encourage you to go check out the talks another aspect is that if you're a partner and you're interested in in in getting in this program we also would like to hear from you talk about learning we've invested a lot in producing quality material to help you learn about ml and about tensorflow one thing that we did over the summers what was really exciting is for the first time we were part of the google Summer of Code we had a lot of interest we were able to select 20 very talented student and they got to work the whole summer with amazing mentors on a tensorflow engineering team and they worked on very inspiring projects going from 2.0 CU Swift's UJS to TF agents so we were so excited with you know the success of this program that we decided to participate for the first time in the Google coding program so this is the same program but for pre-university students from 13 17 is a global online contest and introduced teenagers to the world of you know contributing to open source development so as I mentioned we've invested a lot this year on ml education material but one thing we heard is that there's a lot of different things and you know what you want is to be guided through pathways of learning so we're we've worked hard on that I'm excited to announce the new learn ml page on centerfielder org and what this is is Learning Path curated for you by the 10th of 13 and organized by level so you have from beginners to advanced you can explore books courses and videos help you improve your knowledge of machine learning and use that knowledge and use tensorflow to solve your real world problem and for more exciting news that will be available on the website I'd like to play a brief video by a friend and rank hi everyone I'm in New York right now and wish I could be there to enjoy the conference but I want to share with you some exciting updates D plans are AI started a partnership with the tensorflow team with the goal of making world-class education available for developers on the Coursera platform since releasing the deep learning specialization I've seen so many of you hundreds of thousands learn the fundamental skills of deep learning I'm delighted we've been able to complement that with the tens of flow in practice specialization to help developers learn how to build mo applications for computer vision and opie sequence models and more today I want to share of you an exciting new project that the deep learning di intensive little teams have been working on together being able to use your models in the real world scenario is when machine learning gets physically exciting so we're producing a new folk or specialization called tensor flow data and deployment that will let you take your ML skills to the real world deploy models to the web mobile devices and more it will be available on Coursera in early December I'm excited to see what you do with these new resources keep learning all right this is really cool we you know since we started working on these programs it's been pretty amazing to see hundreds of thousands of people take those courses and the goal of these educational resources is to let everyone participate in the ML revolution regardless of what your experience with machine learning is contribute so great way to get involved is to connect with your GE we now have 126 machine learning GT e--'s globally we love our gd's are amazing they do amazing things for the community this year alone they gave over 400 tech talks 250 workshops they wrote 221 articles reaching tens of thousands of developers and one thing that was new this year is that they helped with Doc's prints so Doc's are really important they're critical right you really need good quality Doc's to work on machine learning and often the documentation is not available in people's native languages and so this is why we went partially with our DS we launched the docs prints over 9,000 API Doc's were updated by members of the tensorflow community in over 15 countries we heard amazing stories of you know the power power outages and power running out and people you know coming back later to finish the docs brain and actually writing Doc's on their phones so if you've been helping with Doc's and thank you if you're in the room of over livestream thank you so much if you're interested in helping translate documentation in your native language please reach out and we'll help you organize a dog's brain another thing that the GD is help with is experimenting with the latest features so I want to call out samvit even an MLG de from Singapore who's already experimenting with 2 point X and CPUs and you can hear him talk later today to hear about his experience so if you want to get involved please reach out through GD e and and start working on sense flow another really great way to help is to join a safe a cig is a special interest group and it helps you work on the things that you're the most excited about on tensorflow we have now 11:6 available add ons yo and networking in particular really supported the transition to 2.0 by embracing the parts of contrib and putting them into 2.0 and sig build ensures that TF runs well everywhere on any OS any architecture and plays well with the Python with the Python library and we have many other really exciting sticks so I really encourage you to join one you another really great way to contribute is through competition and for those of you who were there the dev summit back in March we launched our 2.0 challenge on Def Post and the grand prize was an invitation to this event tensorflow world and so we would like to honor our 2.0 challenge winners and I think we are lucky to have two of them in the room of Victor and Kyle if you're here so Victor worked on handtruck ideas a library for prototyping hand gesture in the browser and then a Kyle worked on a Python 3 package to assimilate n body to generate n body simulation so one thing we heard too during our travels is oh that dad that hackathon was great but I totally missed it can we have another one well yes let's do another one so if you go on TF all deaf boys calm we're launching a new challenge you can apply your 2.0 skills and share the latest and greatest and win cool prizes and so really excited to see what you're going to build another great community that we're very excited to partner with is Kegel so we're a long we've launched the context on cargo to challenge you with a question answering model based on Wikipedia article you can put your natural language processing skills to the test and earn $50,000 in prizes it's open for entry until January twenty seconds so best of luck so we have a few action items for you and they're listed on the slide but remember we created tents of the world for you to help you connect and share what you've been working on so our main action item for you in the next two days is really to get to know the community better and with that I'd like to thank you and I hope you enjoy the rest of TF world thank [Applause]