Google Cloud Applied AI Summit - Generative AI: The next frontier for developers

Key Takeaways

[Music] e [Music] [Music] [Music] oh he [Music] good morning today is a really exciting day for all of us we've been working on AI for years and our goal has been to create a platform that allows people to build applications that can understand reason and act on information in the same way that humans do human beings understand the world through an amazing array of information all at once we see hear read Listen and talk about many different types of information text audio code images and video simultaneously people want to build with AI that can take this reality into account these advances will allow developers and businesses to custom build agents that can reason understand and do more on people's behalf last week we made a massive step forward towards this Vision with Gemini Google's most capable model it's the world's first native multimodal model capable of making sense of text audio code images and video all at once Gemini has significant breakthroughs in advanced reasoning and understanding today we'll show you a glimpse of some of the advances that are made possible by Gemini we will also talk about our new AI hypercomp computer advances in our vertex AI platform that help Dev developers build amazing things with Gemini and new capabilities in Duet AI to help developers and cyber security analysts be more productive let's watch a quick introduction of Gemini Google's largest and most capable AI model if I were to look at the foundational breakthroughs in AI over the past decade Google has been at the Forefront of many of those breakthroughs Gemini is our largest and most capable model it means that Gemini can understand understand the world around us in the way that we do so not just text but also code audio image and video Taylor used Gemini to search a large Cass of scientific papers for key information we wrote a prompt with its Advanced reasoning capabilities Gemini was able to distinguish between papers that were relevant to the study and those that weren't I'm delighted to introduce Alpha good 2 powered by Gemini when we evaluate alphacode 2 on the same platform as the original alha code we solve almost twice as many problem problems Gemini on its own has the ability to transform software development as we understand it but safety and responsibility has to be built in from the beginning and that has oriented us to be both bold and responsible together developers and Enterprise customers are going to figure out really creative ways to further refine our foundational models Gemini will be available in three sizes Gemini Ultra our most capable and largest model for highly complex tasks Gemini Pro our best performing model for a broad range of task and Gemini Nano are most efficient model for on device tasks it's been a Monumental engineering task which has been you know very challenging but also very exciting here to tell us more Eli Collins Google deep Minds vice president of product thanks Thomas Gemini represents a large scale collaborative effort of people from across Google Gemini is our largest and most capable AI model and it's our most General we've rigorously tested Gemini models on 30 of the 32 widely used industry benchmarks our largest and most capable model Gemini Ultra exceeds current state-of-the-art results in fact on one of the most popular benchmarks MML or massive multitask language understanding Gemini is the first AI model to outperform human experts with a score of 90% And notably on the new multimodal benchmark mmu Gemini achieves a state-of-the-art score of over 59% we've optimized Gemini for three different sizes which means it's able to run on everything from mobile devices to Data Centers Gemini Ultra is our most capable and largest model for highly complex tasks Gemini Pro is our best model for scaling across a wide range of tasks and Gemini Nano is our most efficient model for on device tasks so those are its performance stats now let's get into a little more about what makes Gemini such an exciting advancement over existing models Gemini is natively multimodal it has sophisticated reasoning capabilities and can code at an advanced level let's jump in Gemini is built as multimodal from the ground up that means it's trained on different data types simultaneously allowing it to understand and reason across them seamlessly traditionally creating multimodal models involve training separate components for different modalities and then stitching them together to roughly mimic some of this functionality but because Gemini was pre-trained on the start from text images audio video and code it can seamlessly have a conversation across modalities and give you the best possible response Gemini also possesses exceptional reasoning skills it can analyze vast amounts of complex information and extract insights at digital speeds making it valuable in fields like law science and Finance really any industry that wrestles with that challenge another area that's very near and dear to us at Google is Gemini's Advanced coding capabilities Gemini is one of the leading Foundation models for coding in the world Gemini Ultra excels in several coding benchmarks including human eval an important industry standard Benchmark for coding tasks and natural to code our own more reliable coding Benchmark in the future programmers will make use of Highly capable AI models as collaborative tools that can assist with the entire software development process from reasoning about problems to assisting with implementation I've been fortunate to experiment with Gemini for the past few months and I can't wait to see how you all build transformative applications using it I'll hand it back to Thomas to tell you more about how Gemini is coming to life in Google Cloud thank thank you Eli last week we announced Early Access of Gemini Ultra for select customers starting today Gemini Pro is available to all developers and vertex AI early next year we'll also make Gemini Ultra available in vertex Ai and Gemini will be used in Duet AI in both Google workspace and Google Cloud Gemini Pro was designed to be our most efficient model to serve so we've priced it to be accessible to all developers as for price with a focus on hardware and software optimization we've reduced our pricing by four times per input character and two times per output character at the same time improving model quality and reducing serving latency now let's talk about the AI optimized infrastructure on which emini was trained and is being served how developers can access and build agents powered by Gemini and new capabilities we're introducing with duet AI to help developers and security analysts let me start by inviting Amin vat to share more about our AI infrastructure thank you so much Thomas let's take a quick look at some of our groundbreaking a infrastructure Innovations [Music] [Music] from fully integrated AI optimized Stacks to more specific TPU AI supercomputers advancements in AI infrastructure must continue to meet and exceed the Computing demands of our customers in terms of Demand with the growth of gen the number of parameters in large language models has increased by 10x a year for the past 6 years that's a total of a factor of 1 million hundreds of billions if not trillions of parameters bring new levels of sophistication and capability but they also of course bring heightened requirements for training tuning and inference we have seen rapid Innovation Through The Years with the evolution of our TPU AI supercomputers we train Gemini 1.0 at scale on our AI optimized infrastructure using Google's inhouse design tensor processing units or tpus V4 and v5e and we designed Gemini to be our most reliable and scalable model to train and are most efficient to serve on tpus Gemini runs significantly faster than earlier smaller and less capable models for more than a decade these custom tpus have been at the heart of Google's AI high powered products that serve billions of users like search Youtube Gmail Google Maps Google Play and Android they've also enabled companies around the world to train large scale AI models of their own cost effectively last week we were thrilled to announce our most powerful and scalable TPU to date Cloud TPU v5p the PN v5p stands for performance what's pictured here is just seven racks out of the 140 that make up a full TPU v5p supercomputer it features more than twice the flops per chip and is four times the total performance relative to the previous generation TPU V4 Cloud tpv 5p allows our team team to train models with even lower latency which accelerates serving future versions of Gemini it also opens up unprecedented capability for our customers to take on the most demanding workloads these tpus are designed with judicious specialization coupled with application code design powered by a synchronous high bandwidth interconnect supporting communication at 4.8 terabits per second per chip customized liquid cooling For maximum system efficiency and performance per watt Optical circuit switching for fa tolerance and the most efficient scheduling specialized data representation spe specific to ml workloads such as inate high bandwidth memory with 10 times the speed of standard Dam specialized tensor and Spar scores for the most advanced mathematical operations underlying ml compute these advancements as well as many others enable our TPU Ai supercomputers and most recently our TPU v5p to power the next generation of multimodal foundational models but everyone knows a chip is only as powerful as the full stack that surrounds it that's why last week we announced AI hypercomp Compu a groundbreaking supercomputing architecture from Google Cloud that optimizes our integrated stack of performance optimized Hardware including the advancements I just shared that are powering TPU v5p Dynamic and efficient software leading ml Frameworks out of the box like Jacks pie torch and tensor flow and the most versatile Resource Management tools based on Decades of research what's exciting about AI hypercom computer is the system level productivity it drives Computing at the scale of geni Demands efficiencies but this is traditionally done via peac meal component level enhancements which can lead to many bottlenecks AI hypercom computer employs system level code design to boost efficiency and productivity across AI training tuning and serving together AI hypercom computer with TPU v5p enables faster and more scalable gen model training allowing new products and capabilities to reach customers sooner but no two AI workloads are created equal which is why we're investing in a broad portfolio of infrastructure for AI at next this past year we introduced the other member of the fifth gen TPU family TPU v5e also designed with judicious specialization ation and application code design TPU v5e delivers 2.3x better training and 2.7x better inference performance per dollar than TPU V4 it it is our most efficient Co it is our most cost efficient TPU to date we also announced the ga of A3 VMS with Nvidia which achieved three times AI training performance compared to the prior generation of A2 VMS we have tremendous interest from customers for our AI infrastructure more than 50 50% of generative AI startups build on Google cloud and More than 70% of generative AI unicorns are Google Cloud customers customers like anthropic and coher are trusting us with their growing demands for AI infrastructure and with that back to you [Music] Thomas Thanks Amin let's show you now how developers can access and build with g Gemini's API starting today today we're making Gemini Pro available in Google AI Studio a new free web-based developer tool that helps you prototype and launch apps quickly with just an API key when it's time for a fully managed AI platform vertex AI our Enterprise AI platform allows you to customize and tune Gemini while addressing Enterprise needs for security safety privacy data governance and compliance vertex AI helps developers discover models tune models with their own data augment them to include up Toth minute information and take real world actions it allows them to manage and scale models in production and to build search and conversational agents in a low code no code environment today we'll talk about about how we're improving the developer experience in each and every one of these areas with the addition of Gemini vertex AI customers can now choose from over 130 models made available as apis including Google developed models like Gemini and a curated list of open-source and third-party AI models all of which meet Google's strict Enterprise safety and quality standards today we're pleased to announce six new and updated models including Gemini Pro of course let me just touch on a few of them imagine 2 is a major update to Google state-of-the-art image generation model you can now create custom images with improved photo realistic image quality text rendering to easily create images with text text overlays and the ability to generate logos we're also excited to announce General availability of Med LM a suite of medically tuned models this Builds on our Innovations in Med Palm 2 to offer flexibility to healthcare organizations we continue our commitment to an open model ecosystem bringing leading open- Source models Mistral IM bind and deto to vertex's AI model Garden we help protect customers in many ways as well today we're extending our indemnification promise to now include model outputs from Palm 2 and imagine 2 this means if you're challenged on copyright grounds we will generally assume responsibility for the potential legal risks involved we will update our indemnification proc promise to include Gemini just as soon as it's generally available now that you have access to the best models you want to customize them with your data for your brand and for your needs vertex AI offers the most robust set of tuning capabilities of any platform making it easy for developers of all skill levels to customize models for instance in Bank this lets you build agents that can access highly private data securely and tune a model to offer the best client service and with your own brand voice we offer multiple ways to tune Gemini with your data to create your own model optimizing performance and model quality these techniques include prompt design which lets you tune Gemini without ml expertise using simple text prompts supervised adapter based fine tuning which lets you use as few as 100 examples to improve the model's performance you can also use adapter tuning with reinforcement learning with human feedback and you can even do full fine tuning with open- Source models we're also introducing a new capability today called distillation distillation allows you to train your own smaller task specific model with generally much less training data and with lower serving cost and latency than the original model distillation is a technique only available on Google Cloud all approaches to tuning and Distilling models ensure that your proprietary data is safe we provide a comprehensive set of controls designed designed to keep your data secure and private with vertex AI your input data your output data any rlf feedback are yours alone and not accessible by anyone else when you use gemini or any other Foundation model on the model card now Gemini is exceptional at understanding large volumes of information and producing outputs but to be truly useful in the real world Gemini needs access to up Toth minute real world information to contextualize its responses and to take action based on this information vertex AI provides you with augmentation tools to accomplish this with grounding on vertex AI you can improve the accuracy and relevance of Gemini's answers using your Enterprise's own data and using citations to understand the source of its responses we offer out of the boox grounding against an Enterprises structured and unstructured data and we're working with a few early customers to test grounding with the technology that powers Google search another way to equip Gemini and other Foundation models with up-to-date Enterprise information is via retrieval augmented generation or rack a technique that involves fetching data from many different data sources including your own companies enriching prompts with that data to deliver more relevant and accurate responses vertex AI offers a fully managed rag platform with an automated endtoend rag workflow including ingestion retrieval prompt augmentation and citations this can eliminate the need for you to have to write complex custom code to integrate data sources and manage queries we also offer the option to build custom rag Solutions with your choice of vector database and high quality multimodal embeddings once Gemini has the right data you can enable it to take action in the real world with vertex AI extensions extensions can be used to build powerful AI agents that can for example ingest internal code bases and automatically look up evolving security threats in real time balance labor assignments based on real-time utilization open work and deadlines aggregate threat intelligence generate proposals and communicate to security teams for fast incident response and even to book calendar appointments make payments and take other actions on your behalf and now vertex is introducing a new capability function calling which allows developers to provide functions as part of the API call to improve model responses Gemini's output can now be fed to the external tool and API of your choice giving you as a developer more control and flexibility once AI agents are built vertex ai's mlops helps you evaluate manage and deploy large models evaluating the quality of a model's response relative to other models is a significant challenge in deploying models vertex makes this so much easier we're now pleased to announce a new capability called automatic side by side which enables automated AB testing for instance with two different sizes of models or two different custom versions in addition to Google AI studio and vertex we've integrated our AI models including Gemini into popular platforms like collab Enterprise Firebase and flutter with these Integrations any developer can now be an AI developer let's see now a demo of Gemini and vertex AI to together with n shot thanks very much Thomas uh sorry you're feeling under the weather but we're going to make this a fantastic experience for all of you so I'm going to start out by showing you an example of how you can use vertex AI with Gemini I'm going to play the role of a developer working for symol home rentals it's a home rental marketplace where we want to create a listing experience that's easy for hosts how easy we should be able to upload an image or video and be able to get really nice descriptions very quickly let's see how we're going to do that with vertex AI platform vertex AI provides a unified set of capabilities for both predictive and generative AI workloads I can prepare data do model development and deploy and use those models but today as a developer I usually start my journey in what we call the model garden and in the model Garden I have access to our first party models open source models and third party models across a wide range of tasks and modalities but the star of the show today is Gemini so let's see it in action I'm going to go ahead and you could see front and center we have Gemini in the model Garden I'm going to go ahead and click on Gemini and you're going to see that I get an overview a set of use cases that talk about visual information seeking uh being able to do captioning and description and even Advanced reasoning you'll also see that I get information directly inside the platform that tells me exactly how to use this if I'm a coder so I don't have to use the guy if I don't want to I can go straight into my IDE of choice and use the vertex AI SDK right on the fly now in this case I'm going to go and open vertex AI Studio and I'm going to start my use case let me go and copy and paste my prompt here which says generate a description of what you see in this video yes video because Gemini as a multimodal model is able to work with video so I'm going to go ahead and insert a video right here and this is going to be of the home rental that I want to create a listing for let's click on it and actually see it it's a really really beautiful place looks like a very nice kitchen very nice living room and it's a beautiful day we should be able to get a great listing out of this so now I'm asking Gemini to generate a description list all the features of the interior of the house and try to be factual in a tone that would appeal to guests let's go ahead and submit that in the background as you heard from me Aman earlier tpus gpus all the advanced infastructure working to bring a Gemini result and look what we get talks about a beautiful home the open floor plan uh the backyard and uh I get this in a bulleted list now this is great but our style at symbol home rentals is a little more fun a little more zany kind of like me and so uh I'm going to use the capabilities that Thomas just talked about our tuning capabilities to get the model to work more in my style so I'm going to go ahead and head over to tuning and distill the new distillation capabilities we just launched and I'm going to create a tuned model and I'm going to call this tune model symbol home rentals model I'll use Gemini Pro I'll pick an output directory I'll continue all I need is a Json L file with 100 to 500 examples to teach the model how to speak the language of symbol home rentals so I'll go ahead and bring on that Json L file pick my cloud storage location and I can even enable the new model evaluation capabilities Thomas just talked about in a single click now this can take about an hour so uh instead of holding you on the video broadcast live right now for an hour I'm going to go ahead and use a pre-trained model that I created yesterday using the exact same data set and now let's see what it does I'll go ahead and go back into the studio I'll copy and paste that exact same prompt again let's also go ahead and insert the same video that we saw before but now instead of Gemini Pro I'm going to use my tuned symbol home rentals model and let's see if the output looks like what I wanted it to we'll give it a second or two as I does all of its amazing processing and there we go now this is the brand style that we love at symbol home rentals we talk about a sprawling Abode a welcoming Embrace a testament to architectural grer uh that's how our customers love to be able to talk about their properties and now Gemini speaks that way too now of course Gemini speaks many languages 38 as a matter of fact so let's see what happens s if I ask Gemini to go ahead and speak in Spanish I'm going to go ahead and speak in Spanish here I'm going to go ahead and click submit and we will see what Gemini does with Spanish translation there we Goa thank you to my high school Spanish teacher for training me to be able to say this on a live broadcast so this is great this is exactly what I wanted Gemini to do now you should also know that everything I'm doing here in a point-and-click fashion is also available in code it's very easy to be able to use python nodejs or Java it's a symmetric experience across any modality so this is great now I want to teach Gemini to be able to talk to the outside world so let's go ahead and actually show our new extensions capability which we just launched I'll head to the Builder I am going to go and reach out to the symbol publishing API and I'm going to upload an open API spec file call it home rental API operations and what you're going to see is that vertex AI comes back and shows me all of the operations I'm able to do uh with this model and this extension so I can export the description publish listings but I can also get realtime information from the outside world so let me go ahead and create this extension and now I can actually test it uh in fact I'm going to ask a question here um how much should I list this house for uh please provide weekly rates and keep in mind December holidays and let's let's see the realtime capabilities in action of what we can do and look at that the extension comes back in the back end we're using vertex's Vector search capabilities as well as our multimodal embeddings to be able to compare all the different house listings the videos of those house listings and the text and we get back the results December 1st to December 23rd looks like the rates are pretty reasonable and no surprise starting on the 24th the rates go up that means this is doing exactly what I hoped it would now everything that you've seen so far as Thomas talked about is available today when you log in to vertex AI but now I want to show you the art of the possible for what's to come with vertex in the next few weeks we saw an example we go back to our AI Studio of what would happen when we had video let's go ahead and recreate that I'll go and copy and paste this again and I'll go ahead and insert my video but you heard earlier from Eli that this is a model Gemini capable of working with images text and video and doing Advanced reasoning so let's see how that's going to work inside of vertex we saw video before but that was only for the interior of the house let's go ahead and now add the front and the rear of the house in pictures pictures text and video all together simultaneously with Gemini let's go take a look at the pictures there's the front of the house there's the back of the house and here is the same video that we saw before and now I'm going to tell Gemini to generate a description of what you see in this video and these images and in the description list all the features in the interior and exterior of the house and I'm hoping that Gemini will show me that it's able to reason that these are all pictures about the same house inside and outside and come up with an amazing description for my customers let's go ahead and submit that we let Gemini do its magic and look at that picture this a stunning Abode adorned with modern architecture and expansive windows that flood the interior with natural light so Gemini figured out that we were talking about the outside of the house with the images but then now Gemini starts to talk about being able to be greeted by an open floor plan and the living room and the fireplace that's the inside of the house Gemini was able to connect the inside and the outside of the house together across text images and video this is the power of multimodality inside of vertex vertex's Gemini multimodal support for images and video our new tuning capabilities our multilingual support and our extensions they give me all the pieces I need as a developer to make home listings easy for our customers back to you Thomas hey thank you nja that was awesome can't wait to get it in the hands of all developers now let's talk about how vertex AI makes it easy to create search and conversation agents quickly without writing code whether you're looking to generate retrieve or summarize information you're going to need to search your Enterprises data for most generative AI powered applications and agents vertex AI search provides you with a Google quality search experience out of the box information retrieval and answer generation system vertex AI search benefits from Google's deep experience in search including implementing a hybrid of semantic and keyword search understanding user behavior and intent document annotation with a Knowledge Graph and reranking techniques using state-of-the-art model distillation technology now building a Google quality search engine for retrieval augmented generation powered applications is hard vertex AI search makes the entire process of ETL OCR chunking embedding indexing reranking storing retrieval and summarization super simple to today we're pleased to announce three important new advancements in vertex AI search first we will very soon be integrating the power of Gemini into vertex AI search as an option for summarization and answer generation enhancing the quality accuracy and grounding capabilities of your search results second we're pleased to announce Blended search support now in preview in vertex AI search what is Blended search it allows you to run a single query across structured data like your product catalog unstructured data like customer chat logs and forums and public web pages like your product descriptors this means you don't have to build separate search engines for each type and modality of data and then integrate them Blended search means users get better quality search results faster third we're introducing Vector search Vector search is for customers who have more complex use cases and want to build custom search applications using multimodal embeddings for ad serving or e-commerce recommendations we offer tuning capabilities for Vector search using our platform offerings like vector search multimodal embeddings or our embedded Vector database Vector search can now also be used by Gemini to power your advanced search applications vertex AI conversations facilitate the creation of natural sounding humanlike voice and chat-based conversational agents starting today you can choose Gemini as an optional Foundation model for your conversational agents using Gemini's Advanced reasoning to handle more complex questions we're also introducing a few important new capabilities first playbooks which are now in preview playbooks allow you to teach the conversational agent in the same way you would a person give giving it instructions in natural language playbooks leverage vertex ai's broad Suite of connectors and extensions allowing conversational agents to serve fresh information to users and complete actions such as payments booking scheduling meetings on users behalf you can enter the agent Persona goals and the steps you want the agent to follow in simple natural language you can also provide examples of how it's done and you can preview your agent responses by testing it before deploying it second vertex AI conversation now has an inline simulator to improve the quality of the agent responses with full debuggability and a streamlined feedback loop process third conversational agents can be built once and deployed across multiple channels including web mobile app call center and in retail points of sale for example in retail stores or Bank branches we're seeing strong momentum with vertex AI search and conversations for instance helping customers and employees with research assistant food ordering travel planning and so much more let's hear from Forbes who recently launched their adelade search experience I'm David Johnson I'm Chief data officer Forbes media using vertex AI search and conversation Forbes has built Adelaide which is our new interactive search experience so you could have a user that comes to Forbes looking for travel Insight you could type what are the top 10 vacation destinations what are some great hotels around there and it maintains the context vertex AI search and conversation was a very light lift for our data science and ml Ops teams we had a proof a concept up in a matter of 2 weeks that was unprecedented for us with Google Cloud Forbes is building a new way for our users to access the data they want now that we've covered vertex Ai and Gemini together let's look at how we at Google are building our own agent duet AI to assist people at work please welcome Gabe Monroy thanks Thomas earlier this year we unveiled duet Ai and always on collaborator duet aai Works in Google workspace and has been GA since August to help write and refine in Google Docs and Gmail generate custom images in Google slides and more DUI also works in Google Cloud helping with development operations Data Insights cyber security and more today we are announcing that duet aai for developers and duet aai in security operations are both generally available let's start with what we're doing for development teams AI assistance brings huge productivity gains when it comes to software development AI assistance can also improve the quality of the software that's being written for instance by improving documentation of code ensuring that code is better tested and and improving how software dependencies are checked more rigorously with the ga of duet AI for developers we are here to make every Dev team more effective duet a for developers is the only assistant fine-tuned to provide the best possible answers for developers using Google Cloud all with their favorite tools and idees duet AI for developers is also the only AI assistant on the market that goes beyond AI power code and chat assistance to running applications on Google cloud with unique capabilities around application deployment troubleshooting and issue remediation Priyanka let's show how easy it is for developers to build deploy and operate a real world application in less than five minutes with duet AI for developers prianca over to you thanks Gabe there are a lot of capabilities developers expect from a coding assistant things like code explanation and code generation as you would expect D AI is fantastic at these let me quickly show you so here let's ask duet AI to explain me this code and there we have it an entire function explanation for our code now I also want to add unit tests I can do that with just one click see an entire testing scenario is set up for us now I also want to use duet AI to improve the readability of this file by adding documentation and comments directly from my development environment so here I'm going to say add comments to this code and I want to do it for the entire file so here we go and duet aai has generated this entire file again for me with comments now so making it a lot more readable now let's say I want to generate some code so here I am in another file and I've already typed up a comment in natural language which is going to generate a new route a new function for us that's going to take some sales data and our product information and then create a new function for us so here I'm asking duet AI to generate this function and there I have an entire function generated in high quality code now all this is really cool and it makes du AI a joy to use but these are things that you would expect in a coding assistant what makes duet AI unique is that it is the only coding assistant that is native to Google Cloud finetuned on Google Cloud best practices and designed to help you across the entire software development life cycle including running and operating applications now let me show you what I mean so for this code I have the data in a Json file and I would like to store this data in fir store which is Google Cloud's document database now I'm not an expert in fir store but fortunately du AI is so I'm going to ask duet AI chat to respond to me with a code so generate code to create a function that uploads this data into a fire store database and duet AI because it is trained on Google Cloud best practices and products has generated this entire function for me which all I need to do is insert it to my current file and I'm done now I am ready to deploy this web app to Google Cloud so so let's ask D AI to help deploy in a container in Google Cloud which is the home for kubernetes so I am going to ask what are the services to deploy containers in Google cloud and this should be able to give me options that I have to deploy in Google Cloud I have gke I have Cloud run I'm going to select Cloud run because we want to go serverless and now I want to know how do I deploy my application to Cloud run and because we are in Cloud code I'm going to ask it a little more specifically to generate steps for to deploy from cloud code and there we have all our steps and I'm going to follow these steps now and click on deploy and there it is going to try to deploy with After Fall following these steps I have triggered a deployment now this may take a few seconds but duet AI is the only AI assistant that enables a smooth Journey from development to deployment in Google Cloud now while it is taking a few seconds I already have a deployed version um that I've set up for us here so let me click on that one to show you the completed deployment we have deployed our app where you see how much we've sold from these products this month and with that we've built and deployed the app now running and operating applications is equally as important as coding and developing them now to show you what I mean let's check the logs for my running application to see if things are okay so I can go do that in my logs Explorer right here I'm going to click on one of my deployments to open up logs Explorer and here I can see all of my logs but let's say I want to actually drill deeper and perform some Advanced tasks and for that I'm going to click and get into my logs Explorer in Google Cloud console and this is one of the logs that I'm interested in catching my eye because it says insecure request warning and that is not supposed to be good so let's drill a little bit deeper into that and and I see this explain this log entry button so I'm going to click on that and here as well I love how du AI is still by my side here in the console where I'm troubleshooting a live running application now here it looks like we have an insecure https request so we might want to secure that so I'm going to ask some follow-up questions to do at AI to help me fix this code to make it more secure so let's say I I ask how do I secure my in Secure https Connection in a python file now I'm making all these spelling mistakes but I hope D AI catches them and there it has done that and it's telling me a little bit about what I can do which files I can take and I can just it's an easy fix and which will help improve our security quite a bit well with duet AI we have turned our hourlong task of building a real world app in just five minutes my team is going to be so impressed with me back to you Gabe thanks Priyanka so amazing to see a real world app built so quickly what I found most impressive is that duet is the only AI assistant on the market that handles the entire software development life cycle from development to operating a running application with quality at every step with duet AI we're helping leading Brands like PayPal boost developer productivity and we are enabling retailers like GE Appliances to gain new insights for better and faster business results soon customers will be able to fine-tune the underlying models for a duet AI with their own code base so that developers can receive code suggestions informed by their intern internal private code we're currently testing this with a few select Enterprise customers and expect to be generally available in 2024 today we are also announcing more than 29 Cod assist and knowledge-based partners who will contribute data sets specific to their platforms so that users of duet AI can receive AI assistance based on Partners coding and data models their product documentation and best practices CODIS part Partners help du at AI for developers provide technology aware coding assistance with Partners like confluent hashicorp and mongodb duet a for developers will be trained to generate code that is specific to those platforms so that developers can build and troubleshoot faster knowledge-based Partners allow duet AI for developers to provide access to documentation and knowledge sources like Partners such as data dog jet brains and Lang chain lastly Google Cloud Service Partners play a critical role in helping customers adopt Genai including duet aai these Partners have committed to train over 150,000 experts and can help Enterprises brino Ed AI into their existing development workflows next I'd like to talk about Security in Google Cloud we help protect customers applications and businesses end to end first we provide worldclass thread intelligence via Mandan which provides information on the latest attack vectors as well as assessments of who may be targeting our customers second we use that thread intelligence to quickly detect investigate and remediate security incidents through Chronicle across all of the customer environments including on other clouds third for Google Cloud specifically we go deeper via security Command Center that understands most parts of the customers applications running in Google Cloud to identify misconfigurations vulnerabilities and threats and help mitigate and remediate those risks now across each of these we are infusing duet aai into key security workflows which help security customers detect threats faster and reduce the toil for security operations to protect their organizations today with duet Ai and security operations we are the first cloud provider to make gen generally available in a unified SEC Ops platform this incorporates are leading security intelligence to help Security Professionals protect their organization and respond to a barrage of alerts and security anomalies each day with duet a and security Operations Security Professionals can search event data and query across many different log types without needing to know specialized syntax saving value time early user feedback show that duet aai reduces the time analysts spend writing running refining searches and triaging complex cases by approximately seven times Priyanka let's show how you can find security risks and address them quickly with duet AI in security operations prianca over to you thanks Gabe duet AI in security operations helps improve productivity for security teams here I am in chronicle our threat detection and intelligence platform and I'm looking to get some insights into the crypto mining attacks in our environment in the last week so I type my query in natural language and duet AI converts my request into a query that Chronicle can understand from here I can click on search and Chronicle is going deep into understanding all this data and getting me a list of all the alerts around crypto Mining and once it gives me that list there we go here's our list I can actually drill deeper into the these alerts and find out a little more about what is happening in our environment so here I am in a detailed alert View and duet AI is following me along as an AI assistant here as well instantly summarizing case data and alerts for me in including what has happened and why it matters and also recommendations on how to respond next now in the world of security every second counts these are critical details any security practitioner needs whether they are new to the team or a seasoned SE Ops analyst back to you Gabe thanks Priyanka incredible duet AI in security operations launched in preview earlier this year with over 100 50 customers feedback has been overwhelmingly positive with less than 1% of requests receiving a negative rating by users customers like fizer are excited about DUI's potential and the ability to use Simple language queries to search pedabytes of data and perform complex analysis onboard analyst faster and rapidly pursue Advanced threats to get started on duet AI please explore the security operations Enterprise and Enterprise Plus packages you can get started at no cost on duet AI for developers until February 1st 2024 by visiting our website and stay tuned for more to come in 2024 before I hand it back to Thomas let's hear from Turing who's been using duet aai to improve developer productivity I'm Jonathan sadar CEO and co-founder of touring touring is the world's first a AI powered tech services company with over 3 million software Engineers on our platform we use AI to automatically vet them match them and to speed up the productivity of the software Engineers that we deploy to companies tools like do it AI help us give superpowers to every software engineer it helps us ship higher quality work FAS we saw productivity improv by over 30% I just think of what a planet of 7.8 billion people could do with 10x more productivity it's going to be a Wonderful World hey thank you Gabe and pranka along with our AI technology stack we've been on a mission to build the most Innovative and open ecosystem for customers to build AI agents this includes Foundation model Partners like AI 21 Labs meta mistol tools and applications Partners like sap and day our network of Professional Services Partners helps customers successfully build and bring generative AI to their organizations one of our largest Professional Services Partners globally is Accenture today I am pleased to announce an expanded partnership with Accenture to deliver generative AI to Enterprises to improve operations create new lines of business and to build amazing and unique customer experiences we'll provide engineering and rapid prototyping services for generative AI including Gemini we're also pleased to be working with a large number of customers who are using and building with our generative AI models customers like AI 21 bending spoon lyrix app Lovin and AI assembly AI a trusting us with the growing demands for AI infrastructure customers including McDonald's carfor Shutterstock Bayer Spotify dun and Brad street Formula E the World Bank Yahoo and Jet brains are using vertex and our models to build their own agents and L'Oreal Liberty Global purple soil and ax for instance are using duet AI to increase the productivity and resolve security threats faster we're very pleased to announce a new partnership today with Hashi cop our teams will work together to launch new AI powered features across Hashi Cobb's product Suite all built on vertex AI duet AI will also make it easier for Google Cloud users to use terraform today's announcements wrap up a super busy year for our engineering teams here are some statistics for you since January we've introduced a number of important new capabilities in our AI hyper Computing infrastructure with new advances in gpus tpus ml software compilers Frameworks workload management and many others we've introduced three new major generations of models culminating of course with the introduction of Gemini today with over 500 features in vertex Ai and a new suite in Duet AI in both workspace and in Google Cloud we've seen amazing unbelievable developer growth for example just last quarter the number of active generative AI projects in vertex AI grew by seven times in just one quarter we're wrapping up the year with a slate of new advances today we introduced Gemini a state-of-the-art model that delivers a variety of state-of-the-art features including native multimodality and advanced reasoning we introduced a new AI hyper computer that leads the world in performance latency and cost in both AI training and serving we brought Gemini to vertex AI starting with Gemini Pro along with significant new features to build agents that combine native multimodality Advanced reasoning up to the minute knowledge and to take actions all fine-tuned to your data with duet AI for developers we help you develop document test refactor and modernize code while working within the development tools and platforms that each of you developers are familiar with we also released two at AI in security operations to help protect you and your company from cyber threats we've updated pricing that makes Gemini accessible to every developer and we're introducing expanded indemnification to help protect you from copyright concerns we continue to take important steps to enable developers to build agents that think understand and act on information the way that we as humans do at the same time we remain firmly committed to our promise to be bold and responsible in bringing you these advances thanks for joining us today next you can get a Hands-On Deep dive with our developer Advocates Nikita and Dale and then dive into breakouts for app building data science and great partner Solutions thanks and enjoy the rest of our Summit and that's rest he [Music] [Music] oh [Music] [Music] hi everyone I'm Nikita and I'm Dale and we're generative AI developer Advocates at Google today you heard about how generative AI is a really powerful tool for developers so now we're going to take a look at how you can get started building Hands-On with Google's multimodal model Gemini let's dive in generative AI allows us to solve all sorts of problems that were difficult for computers to tackle before and one of them is providing answers for complexed nuanced questions phrased in natural language you can think of this as a model's ability to explain something for example if you've ever asked an llm a question like why isn't this code working or how can I explain a combustion engine to a 5-year-old you've experienced this capability firsthand but what's really neat about Gemini is its multimodal support which allows us to ask questions that are just too hard to put into words that we need a picture or a video to explain like I can send Gemini this picture with the text what is this object and how is it used Gemini will recognize that this is a sun dial a device that uses the sun's position to tell the time another great way to use this feature helping users troubleshoot not just software but also life like like let's say that I want to build an app that helps people take better care of their house plants and troubleshoot when something goes wrong I want to be able to upload a picture of a plant that's not looking too great and know what I should be doing to keep it healthy let's prototype this functionality with Gemini so I'll jump into vertex AI studio and I'm going to open up a new prompt and upload an image the image of my plant and then I'll say identify this plant and what's wrong with it I'll hit submit and I'm goin