What's new in TensorFlow

Key Takeaways

hello and welcome to tensorflow and Keras at Google I O I'm mani vardarajan and I'm the director of engineering for Google's ml framework apis in this session we're going to take a quick look at some of the many improvements and additions coming your way this year in tensorflow and the high level modeling Library Keras here's what we'll cover today first we'll show you how Keras CV and Keras NLP give you access to pre-trained state-of-the-art models in just a few lines of code so you can innovate and explore freely then we'll talk about detensor using detensor you can easily scale up your models and train them efficiently by combining different parallelism techniques we'll also share Jax to TF an exciting new feature that allows the high performance Jacks framework to leverage the incredibly powerful and diverse tensorflow ecosystem and finally we'll look ahead to a preview of our new quantization API which will enable you to make your models more cost and resource efficient without compromising on accuracy now why are each of these so important well let's take a look at the landscape the world of machine learning is changing faster than ever over the last few years we've seen models go from hundreds of millions to hundreds of billions of parameters and as models have gotten bigger training and deploying them has gotten more complex and more expensive models are also deployed in more places ideas that once started in research are now used in production on billions of mobile devices and Beyond and tensorflow is where so much of it happens we are the largest ml development community in the world we take immense pride in this we also want to continually make our ecosystem better for everyone let's get right into it starting with Kara CV and Keras NLP Keras CV and Keras NLP are powerful modularized libraries that give you direct access to the state of the art in computer vision and natural language processing whether you want to classify images auto-generate text from prompts like Bard or anything in between keroscv and Keras NLP have got what it takes and it's as simple as it gets Kera CV and Keras NLP provide The Cutting Edge backbone with just a few lines of code and since it's part of Keras it's fully integrated with the tensorflow ecosystem and backed by quality documentation so you can focus on what matters most innovating here's a quick look at some code in less than five lines you can bring the state of the art to life we have a lot more examples like this if you want to dive in deeper check out our full talk on kerascv and Keras NLP Linked In the description below now let's take a look at detensor and how we're enabling you to create ml models at unparalleled scale we're all seeing it models are getting huge as models get bigger training and serving them gets even more complex that's where detensor comes in today we're sharing a simple toolkit to build models at a scale like never before and it's designed to meet the needs for all in our community detenser is built to be flexible efficient and device agnostic let's zoom in to see how it all works traditionally ml developers have scaled up their models through data parallelism which looks like splitting up your data and feeding it to horizontally scaled model instances now standard data parallelism does successfully scale up training but with a big caveat it all has to fit within a single device as models get bigger this is no longer a guarantee you need to be able to scale your models across devices as you can see here detensor allows you to safely Shard across multiple devices but it doesn't just end there d-tensor also enables model parallelism which looks like sharding the model itself across multiple devices and feeding in full copies of your data to each shard but here's what makes detensor really special with detensor you can paralyze your data and your model all in one place the principles behind sharding your model for use with data replicas or sharding your data for use with modeled replicas all still work exactly the same way and come together seamlessly Under One Roof through the power of detensor all you have to do is take any model as you've already written it and add just a few lines above to set up an appropriate config and initialize the detensor context that's it if you attempted traditional data parallelism for the model in this example it would have errored outright the model weights are just too big for a single device but with detensor this isn't a problem you don't have to worry about rewriting your model or writing different code for different parallelism strategies whether you're using one device or 100 detensers got you covered and it's only getting better performance today is already on par with state-of-the-art industry benchmarks and we already have improvements in the works to surpass those baselines we also want to give you a quick preview of what lies ahead detensor will be fully integrated with key interfaces like TF distribute and Keras as a whole with one entry point regardless of device and a number of quality of life features if you want to learn more visit tensorflow.org or check out the integration guides on keras.io another area we'll be covering today is what we're doing about supporting our research Community better especially with the emergence of jacks many of the ml advancements that are now household names had their Beginnings in research this includes the latest models in the news like Bard and chat GPT these architectures came from Google's own published research Jax has emerged as a trusted tool for much of the research Behind These models but productionizing research is hard and so we've been putting a lot of thought into how we can bring to production power of tensorflow's ecosystem to Jax to that end we're very excited to introduce Jax to TF which provides a clear pathway from Jax to the tensorflow ecosystem Jacks to TF is a simple lightweight API and there's a lot you can do with it for one you can take Jax to TF to take a Jax model and deploy it either on a server using TF serving or on device using TF Lite you can also use jackstatia for fine tuning by taking a pre-trained Jax model into tensorflow and continue training it from there you can even fuse models together by taking a Jax model merging it with additional layers and other components and then train them as one model in tensorflow this is a very powerful capability so let's see how it works at a high level to bring a Jax model into tensorflow you could do something like the following first you would Define a model in Jax second you would create a basic wrapper class which uses jax2tf.convert to express Jax methods as tensorflow functions and that's pretty much it you can save it into a tensorflow saved model and directly serve fine tune fuse and more just like we've talked about it's incredibly easy to do and your models will still converge quickly and accurately using Jax to TF is a powerful way to combine Jack's and tensorflow to accelerate research to production we look forward to seeing how the community uses it and with that let's close with a sneak peek into our efforts to help you make your models more efficient and easier to deploy across a wide variety of devices so how are we doing this with the tensorflow quantization API which will be available later this year quantization is a set of techniques that allow you to reduce model size making your models run faster and consume fewer resources this means reducing how much memory and compute you require to run the models this can reduce things like Mobile Battery consumption or server latency and infrastructure costs earlier versions of quantization toolkits including those in tensorflow were limited but the tensorflow quantization API expands far beyond anything that's come before it first it's more flexible while our previous version was limited to mobile and required using TF Lite this API enables quantization everywhere including server mobile embedded Etc second it's easier it works right out of the box with simple configurations requiring no changes to model code whatsoever third it's also more efficient giving you the power to quantize per layer per op or even per tensor to build in a way that works best for you let's see it in action in just a few simple lines we can prepare the model for quantization and train or save it within a quantization context because our new quantized model is naturally compatible with the rest of the tensorflow ecosystem we can now harvest the fruit of quantization and that fruit is sweet indeed we ran a bunch of tests using the mobilenet V2 model on the pixel 7 and saw nearly 17 times gain in serving throughput versus the non-quantized CPU Baseline but here's the best part all that benefit comes without any noticeable negative impact on accuracy and that's just the beginning the TF quantization API isn't public just yet but will be available soon and we'll continue to evolve it to provide even more benefits and that's a wrap today we've shown you just a few of the key things we've been working on and there's a lot more to come we can't wait to see what you will build and we're always inspired by our community's enduring enthusiasm and continued partnership thanks for stopping by [Music] foreign [Music]