Microsoft Build opening keynote | KEY01

[Music] [Music] [Music] [Music] I think that our industry has to have a common Vision it was a time that connected us to incredible things my name for this vision is information at your fingertips and three decades later we find find ourselves in a new era one where access to information becomes access to expertise from the farm to the lab from The Boardroom to the classroom this new generation of AI is for everyone everywhere now anyone can save time with a personal assistant with G of go ballot I'm saving about 50% of time and that's time that I can use to do other Innovative things anyone can access a personal tutor to learn new skills I think this technology has a potential to completely reimagine the way every single student learns in the world this is a new way to analyze with a personal we're going to be able to have not only productivity gains but insights served to Us near real time generative AI can learn from the data to help improve the farmer productivity AI is unlocking creativity for us all descriptions are so detailed in my imagination I can pains the artwork with expertise at your fingertips you can build what [Music] matters welcome to the age of AI [Music] transformation good morning good morning it's fantastic to be back here at Microsoft build welcome to everyone here and joining us on the web you know developer conferences are always most exciting most fun when there's these fundamental changes that you can sense in the air uh you know I've marked all my adult life uh by coming to pdc's and bills you know for the last three decades I still remember you know distinctly the first time uh win32 was discussed I guess it was 91 uhet Azure right these are moments that I marked my life with and it just feels like we're yet again at a moment like that it's just that the scale the scope is so much deeper so much broader this time around right every layer of this Tech stack is changing you know from everything from the the power draw and the cooling layer of the data center to the npus of the edge are being shaped by these new workloads right these distributed synchronous data parallel workloads are reshaping every layer of the Tex stack but if you think about even going all the way back to the beginning of modern Computing say 70 years ago there have been two real dreams we've had first is can computers understand us instead of us having to understand computers and second in a world where we have these ever increasing information um that of people places and things right so as you digitize more artifacts on people places and things and you have more information can computers help us reason plan and act more effectively on all that information those are the two dreams that we've had for the last 70 plus years and here we are I think that we have real breakthroughs on both fronts the core underlying force one of the questions I always ask myself it's like okay this is great this is like maybe the Golden Age of systems what's really driving it I always come back to the scaling laws just like Mor's law you know helped Drive the information revolution the scaling laws of dnns are really along with the model architecture interesting ways to use data generate data um are really driving this intelligence Revolution um you could say Mo's law was probably you know more stable in the sense that it was scaling at maybe 15 months 18 months uh we now have these things that are scaling every 6 months or doubling every you know 6 months you know what we have though with the effect of these scaling laws is a new natural user interface that's multimodal that means supports text speech images video as input and output we have memory that retains important context recalls both our personal knowledge and data across our apps and devices we have new reasoning and planning capabilities that helps us understand very complex context and complete complex task right while reducing the cognitive load on us but what stands out for me as I look back at this past year is how you all as developers have taken all of these capabilities and applied them quite frankly to change the world around us right I'll always remember this moment in January 2023 uh when I met a rural Indian farmer who was able to reason over some government Farm subsidies that he had heard about on television using GPD 35 and his voice uh if it was remarkable right for me it just brought home the power of all of this because the a Frontier Model developed on the west coast of the United States just a few months earlier was used by a developer in India to directly improve the life of a rural Indian farmer the rate of diffusion is unlike anything I've seen in my professional career and it's just increasing in fact earlier this month I was in Southeast Asia I was in Thailand where I met a developer and I was having a great roundt he was talking to me about how he's using 53 and gbd4 uh and he was using 53 to just optimize all of the things that he was doing with rag I mean this is crazy I mean this is unbelievable it had just launched a few weeks earlier and I was there in Thailand in Bangkok listening to a developer talk about this technology as a real expert on it so it's just great to see the democratization force that we love to talk about but to witness it is just been something and this is quite frankly the impact of why we are in this industry and it's what gives us I would say that deep meaning in our work so I want to start though with a very big thank you to every one of you who is really going about bringing about this uh impact to the world thank you all so very much you know when I think about what progress we've made even since last time we were here at build we've built really three platforms the first is Microsoft co-pilot which is your everyday companion it puts knowledge and expertise at your fingertips helps you act on it and we built the co-pilot stack so that you can build your AI applications and solutions and experiences and just yesterday we introduced a new category of copilot plus PCS the fastest AI first PCS ever built all three of these things are exciting um platforms but I want to start with co-pilot plus PCS you know we are exposing AI as a first class name space for Windows this week we are introducing the windows co-pilot runtime to make Windows the best platform for you to be able to build your AI applications yeah you know what win32 was to grab iCal user interface we believe the windows co-pilot runtime will free for AI it starts with our Windows co-pilot Library a collection of these ready to use local apis that help you integrate into your new experiences all of the AI capabilities that we shared yesterday now this includes no code Integrations for Studio Effects things like creative filters teleprompter voice focus and much more but of course you if you want to access these models itself right you can directly call them through apis we have 40 plus models available out of the box including F silica our newest member of our small language family model which we can specific which we specifically designed to run locally on your npus on copilot plus PCS bringing that lightning fast local inference to the device you know the other thing is the co-pilot Library also makes it easy for you to incorporate rag inside of your applications on with on device data right it gives you the right tools to build a a vector store within your app it enables you to do that semantic search that you saw with recall but now you can in your own application construct these prompts using local data for rag applications now I'm so thrilled to announce as well today that we will be natively supporting pytorch and new web NN framework through Windows Direct ml you know native py support means thousands of OSS models will just work out of the box on Windows making it easy for you to get started uh in fact with web andn web developers finally have a web native machine learning framework uh that gives them direct access to both gpus and npus in fact last night I was playing with it turning on an edge and seeing that uh the web NN uh sample code running it's just so cool to see it you know now use even the npus both py pytorch and webin are available in developer preview today let's take a look [Music] and these are just one of the many announcements today we are introducing more than 50 plus new products and Partnerships to create up you know new opportunity for you we've always been a platform company and our goal is to build the most complete endtoend stack from infrastructure to data to tooling to the application extensibility so that you can apply the power of this technology to build your own applications and so today I want to highlight our top news for this event across every layer of this co-pilot stack so let's Dive Right In with infrastructure you know we have the most complete scalable AI infrastructure that meets your needs in this AI era we're building azure is the world's computer we have the most comprehensive Global infrastructure with more than 60 plus data center regions more than any other cloud provider over the past year we've expanded our data center regions and AI capacity from Japan to Mexico from Spain to Wisconsin uh we're making our best-in-class AI infrastructure available everywhere and we're doing this with a focus on delivering our cloud services sustainability in fact we're on track to meet our goal to have our data centers powered by 100% renewable energy by next year yeah you know we're we're optimizing power and efficiency across every layer of the stack from the data center to the network our latest latest data center designs a purpose built for these AI workload so that we can effectively and responsibly use every megawatt of power to drive down the cost of AI and the power draw uh and we are incorporating Advanced Data Center cooling techniques to fit the thermal profile of the workloads and match it to the environment and the location where it operates um at the Silicon layer we are dynamically able to map workloads to the best accelerated AI Hardware so that we have the best performance uh and our custom IO hardware and server designs allow us to provide dramatically faster networking remote storage and local storage threr you know this end to endend approach is really helping us get to the unprecedented scale in fact last November we announced the most powerful AI simple computer in the cloud for training using just actually a very small fraction of our Cloud infrastructure and over the past six months we've added 30 times that supercomputing power to azure yeah it's crazy to see the scale and of course we're not just scaling training uh fleets we're scaling our inflence Fleet around the world quadrupling the number of countries where Azure AI services are available today and it's great to see that at the heart at the heart of our AI infrastructure are the world's most advanced AI accelerators right we offer the most complete selection of AI Accel accelerators including from Nvidia and AMD as well as our own Azure Maya all dynamically optimized for the workloads uh that means whether you're using Microsoft co-pilot or building your own co-pilot apps uh we ensure that you get the best accelerator performance at the best cost for example you know you see this in what has happened with gp4 right it's 12x cheaper and 6X faster since it launched uh and that's you know the type of progress you can you know continue to see how you know to continue to see the progress as we Evol the system architecture it all starts though with this very deep deep partnership with Nvidia which spans the entirety of the co-pilot stack across both all of their Hardware Innovation as well as the system software Innovation together we offer Azure confidential Computing on gpus to design to be really help you protect sensitive data around the AI model end to end uh we're bringing in fact the latest h20s to Azure later this year uh and we will be among the first Cloud providers to offer nvidia's Blackwell gpus b100s as well as gb200 configurations and we continuing to work with them to train and optimize both large language models like GPT 40 as well as small language models like the 53 family now beyond the hardware we are bringing nvidia's key Enterprise platform offerings uh to our cloud like the Omniverse cloud and dgx Cloud to Azure with deep integration with even the broader Microsoft cloud for example uh Nvidia recently announced that their dgx cloud integrates natively with Microsoft fabric that means you can train those models using djx cloud with the full access to fabric data and Omniverse apis will be available first on Azure for developers to build their Industrial AI Solutions we're also working with Nvidia Nim industry specific developer services and making them fantastic on Azure so lot of exciting work uh with Nvidia now coming to AMD I am really excited to share that we are the first Cloud to deliver General availability of VMS based on AMD Ami 300X AI accelerator it's a big milestone for both AMD and Microsoft we've been working at it for a while and it's great to see that today as we speak it offers the best price performance on GPT 4 inference and we'll continue to move forward uh with azim my in fact our first clusters are live and soon if you're using co-pilot or one of the Azure open AI Services some of your prompts will be served using Maya Hardware now Beyond AI our N2 systems optimization also makes Cloud native apps and the development of cloud native apps better right six months ago is when we announced our first general purpose arm-based compute processor Microsoft Cobalt and today I am really excited to announce the public preview of cobalt-based VMS you know Cobalt is being used for video processing and permissions Management in Microsoft 365 helping power billions of conversations on services like Microsoft teams already and we're delivering that same arms based performance and efficiencies to many customers in fact including elastic mango semen Snowflake and teradata in our most recent Benchmark data and tests our Cobalt 100 VMS delivered up to 40% better performance than any other generally available Arm based VM so we are very very excited excited about Cobalt getting into the market now let's move up the stack to the foundation models look with the Azure AI we offer the broadest selection of Frontier and open- Source models including llms and slms so you can choose the model that makes the most sense for your unique needs and your application needs in fact more than 50,000 organizations use Azure AI today yeah it's great momentum and it also start though with our most strategic and most important partnership with open AI just last week openai announced GPT 40 their latest multimodal model which was trained on Azure it's an absolute breakthrough it has text audio image and video as input and output it can respond and just have a a humanlike conversation that's fast and fluid it can even be interrupted mid sentence uh gbd 40 is also the top performing model on benchmarks across a variety of modalities and it's always going to get you to the best answer uh it has state of the art performance at understanding the information you actually provide in your prompt because that's really what it matter what matters uh what openi demoed last week I mean that was just pretty magical stuff and yesterday we showed how co-pilot will leverage GPT 40 in the video I'm just going to play notice that you can share your screen or session as a prompt in co-pilot and have co-pilot help you in whatever it is that you're doing let's roll the video hey co-pilot how's it going hey I'm doing great and it looks like you're about to dive into some Minecraft ready to build explore and maybe dodge a few moms yeah I'm trying to figure out how to make a sword my son plays this game all the time and I have no idea what I'm doing no worries I'm here to help you impress your son I see you've got a crafting table in front of you which is a great start to craft a sword you'll need some materials can you open your inventory just press e on your keyboard yeah do I have what I need let's see you've got some sticks perfect for the swords handle but it looks like you're missing the material for the blade you can use wood Stone iron gold or Diamond let's start by gathering some wood or shall we head out and find some resources yeah that sounds great oh my God what's that oh no it's a zombie run you need to get away from it fast either Sprint away or quickly build a tower of blocks straight up to get out of its reach if you can find a shelter or dig into the side of a hill do it now just make sure it can't reach you all right I think I might be safe now that was a close one great job finding shelter hey it's really cool to see that you know the magic here is that co-pilot is your personal assistant that's helping you with any task helping you stay quite frankly in the flow in the moment whether it's a game or a document or a you know line of code and as open AI innovates our promise is that we will bring all that Innovation to Azure 2 in fact the same day that open a announced GPD 40 we made the model available aailable for testing on azzure open AI service and today it I'm excited to say that it's generally available on Azure AI you know what this means though is that now we can have these groundbreaking apps that all of you can build using this capability one of the coolest things is that now any app any website can essentially be turned into a full multimod full duplex conversational canvas let's take a look all right let me ask the agent here hey Seth how's it going what can I help you with today I am fantastic I'm sorry I'm in such a hurry I'm going to go on an overnight camping hike where we're camping all this it's going to be dark cold should I should I use these shoes it's the mountains it's probably going to be cold he no worries let me take a look ah those are the trar hiking sandals while they're fantastic for summer hikes and keeping your feet cool they might not be the best choice for oh my okay I'm sorry to interrupt you pick the best thing that I should use and can you just add it to my cart just one product no worries let's go with the Tre ready Hawking boots they're durable provide great ankle support and most importantly they'll keep your feet warm in the cold mountains I'll add them to your cart now ready for check out I see that amazing while Seth was you know Seth was in a hurry but there you go GPD 40 kept up with him it's incredible I really want to thank the open AI team uh for their partnership and and really the responsible approach to Innovation helping our industry move forward Sam will be here in fact joining Kevin in a little bit to talk a lot more about what's coming right because that's the exciting stuff how do you all sample what comes next uh we also have brought we bringing lots and lots of other models as well from coher and data bricks and Desi meta Mistral snowflake all to Azure AI we want to support the broadest set of models from every country every language I'm excited to announce in fact we're bringing models from coher g42 ntity data Nixa as well as many more as models as Services because that's the way you can easily get to managed AI models uh and we all love open source too in fact two years ago uh at Bild we were the first to partner with hugging face making it simple for you to access the leading open source Library uh with state-of-the-art language models via Azure Ai and today I'm really excited to announce announc that we're expanding our partnership bringing more models from hugging face with text generation inference with text embedding inference directly into azur I studio and and we're not stopping there we are adding not just large language models but we're also leading the small language Revolution small language model Revolution you know our 53 family of slms are the most capable and most cost effective they outperform models of the same size with the next size up even across a variety of language reasoning coding as well as math benchmarks uh if you think about it by performance to parameter count ratio it's truly Best in Class uh and today we are adding new models to the 53 family uh to add even more flexibility across that quality cost curve uh we're introducing 53 vision of 4.2 billion parameter multimodal model with language and vision capabilities it can be used to reason over real world images or generate insights and answer questions about images as you can see right here yeah and we're also making a 7 billion parameter 53 small and a 14 billion parameter 53 medium models available uh with f you can build apps that span the web your Android iOS windows and the The Edge uh they can take advantage of local hardware when available and fall back on the cloud when not simplifying really all of what we as developers have to do to support multiple platforms using one AI model now it's just awesome to see how many developers are already using 53 to you know do incredible things from am Solutions the Thai company uh that I mentioned uh earlier the ITC which has been uh built a co-pilot for Indian Farmers to ask questions about their crops epic uh in healthcare which is now using fi to summarize complex patient histories more quickly and efficiently uh and another very very cool use case is in education uh today I'm very thrilled to announce a new partnership with KH academy uh we'll be working together to use 53 to make math tutoring more accessible and I'm also excited to share that they'll be making K Amigo their AI assistant free to all you us teachers let's roll the [Applause] video you forgot already felt like I was in a place in my teaching career where I felt like I was kind of losing my Sparkle and I would just feel really defeated when I looked out on the classroom and I would see students that just didn't look engaged teachers have an incredibly hard job and what we think we can do is leverage technology to take some of the stuff off of their plate to really actually humanize the classroom by some miracle we became a kigo pilot school with new advances in generative AI we launched kigo the point is to be that personalized tutor for every student and to be a teaching assistant for every teacher I started to build these more robust lessons and I started to see my students [Music] engage we're working with Microsoft on these fi models that are specifically tuned for math tutoring if we can make a small language model like fi work really well in that use case then we would like to kind of shift the traffic to fi in those particular scenarios using a small language model the cost is a lot lower we're really excited that kigo and especially in the partnership with Microsoft being able to give these teacher tools for free to us teachers is going to make dramatic impact in US education I think we're going to make them the innovators the questioners isn't that really just why you wake up every morning right because that's our for our future our next generation and to me that's [Music] everything you know I'm super excited to see the impact uh this all will have and what Khan Academy will do and Sal's going to in fact join Kevin been soon to share more and I'm really thankful to teach for teachers like Melissa and everything that they do thank you very much you know of course it's about more than just models it's about the tools you need to build these experiences uh with Azure AI Studio we provide an endtoend tooling solution to develop and Safeguard the copal apps you build uh we also provide tooling and guidance to evaluate your AI models and applic ations for performance and quality which is one of the most important tasks as you can imagine with all these models uh and I'm excited to announce that Azure AI studio now is generally available it's an endtoend development environment to build train and fine-tune AI models and do so responsibly it includes rebuilt in support what is perhaps the most important feature which is in this age of AI which is AI safety Azure AI Studio includes the state-of-the-art safety tooling you know to everything from detecting hallucinations in model outputs risk and safety monitoring uh it helps understand which inputs and outputs are triggering uh content filters uh prompt Shields by the way to detect and block these prompt injection attacks um and so today we are adding new capabilities including custom categories so that you can create these unique filters for promps and completion with rapid deployment options which I think is super important as you deploy these models into the real world when an emerging threat is you know appears Beyond Azure AI Studio we recognize that there are Advanced applications where you need much more customization of these models for very specific use cases and today I'm really excited to announce that Azure AI custom models will come giving you the ability to train a custom model that's unique to your domain to your data that's perhaps proprietary uh that same Builders and data scientists who have been working with openai brought all the FI advances to you will work you know with all of you to be able to build out these custom models the output will be domain specific it'll be multitask and multimodal uh Best in Class as defined by benchmarks including perhaps even specific language proficiency that may be required now let's just go up the stack uh to data ultimately in order to train fine tune ground your models you need your data to be in its best shape and to do so we are building out the full data estate right from operational stores to analytics in Azure we've also added AI capabilities to all of our operational stores whether it's Cosmos DB or SQL or postra SQL uh the core though is this of the intelligent data platform is Microsoft fabric uh we now have over 11,000 customers including leaders in every industry who are using fabric it's fantastic to see the progress with fabric you get everything you need in a single integrated SAS platform it's deeply integrated at most fundamental level with computer and storage being unified your experience is Unified governance is Unified and more importantly the business model is Unified and what's also great about fabric is that it works with data anywhere right not just on Azure but it can be on AWS or on gcp or even on your on premise data center uh and today we are taking the next step we're introducing realtime intelligence in fabric customers today have more and more of this real-time data coming from your iot system your Telemetry systems in fact Cloud applications themselves are generating lots of data but with fabric anyone can unlock actionable insights across all of your data estate let's take a look introducing realtime intelligence in Microsoft fabric an endtoend solution empowering you to get instant actionable insights on streaming data at its heart lies a central place to discover manage and consume event data across your entire organization with a rich govern Lear experience get started quickly by bringing in data from Microsoft sources and across clouds with a variety of out of thebox connectors route the relevant data to the right destination in fabric using a simple drag and drop experience explore insights on pedabytes of streaming data with just a few clicks elevate your analysis by harnessing the intelligence of co-pilot in Microsoft fabric using simple natural language make efficient business decision in the moment with real time actionable insights and respond to changing Landscapes proactively allow users to monitor the data they care about detect changing patterns and set alerts or actions that drive business value all your data all your teams all in one place this is Microsoft [Music] Fabric and we're making it even easier to design build and interoperate with fabric with your own applications right right and in fact we're building out a new app platform with fabric workload development kit so that people like ezri for example have you know who have integrated their spatial analytics with fabric so that customers can generate insights from their own location data using EZ's Rich tools and libraries right on fabric right this is just exciting to see it's the first time you know where the analytic stack is really a first class app platform as well and Beyond fabric we're integrating the power of AI across the entirety of the data stack there's no question that rag is core to any AI powered application especially in the Enterprise today and Azure AI search makes it possible to run rag at any scale delivering very highly accurate responses using the state-of-the-art retrieval systems in fact chat GPT supports for you know where gpts their assistance API are all powered by Azure AI search today uh and with builtin one Lake integration Azure AI search will automatically index your unstructured data too and it's also integrated into Azure AI Studio to support bringing your own embedding model for example and so it's pretty incredible to see Azure search grow over the last year into that very core developer service now let's go up to developer tools nearly 50 years after our founding as a developer tools company here we are once again redefining software development right GI up co-pilot was the first I would say hit product of this generative AI age uh and it's the most widely adopted AI developer tools 1.8 million Subs across 50,000 organizations are using it and get up Coop with get up copilot we are empowering ing every developer on the planet to be able to access programming languages and programming knowledge in their own native language uh think about that any person can start programming whether it's in Hindi or Brazilian Portuguese and Jo they bring back the joy of coding to their native language uh and with co-pilot workspace staying in your floor has never been easier uh we are an order of magnitude closer to a world where any person can go from idea to code in an instant uh you start with an issue it creates a spec based on its deep understanding of your codebase it then creates a plan which you can execute to generate the code across the full repo that is multiple files at every point in this process from the issue to spec to plan to code you are in control you can edit it and that's really what is fundamentally a new way of building software and we're looking forward to making it much more broadly available in the coming months and today we're taking one more big leap forward you know we are Bridging the broader developer tools and services ecosystem with co-pilot for the first time we're really thrilled to be announcing get up co-pilot extensions now you can customize get up co-pilot with capability from third party Services whether it's Docker Sentry and many many more and of course we have a new extension for Azure 2 GI up co-pilot for Azure uh you can instantly deploy to Azure to get information about your Azure resources uh just using natural language uh and what co-pilot did for coding we're now doing for infra and Ops uh to show you all this in action here is n from our GitHub team n take it away thanks seia GitHub co-pilot gives you suggestions in your favorite editor like here where I'm writing unit tests co-pilot is great at meeting you where you're at regardless of the language you're most comfortable with so let's ask for something simple like how to write a prime number test in Java but let's Converse in Spanish using my voice look at that gracias co-pilot co-pilot is great at turning natural language into code and back again but what about beyond the code with the new GitHub Cod pilot extensions you can now bring the context from your connected systems to you so now I can ask Azure where my app is deployed I could ask what my available Azure resources are or I could diagnose issues with my environment and this isn't just for Azure as suia announced any developer can now create extensions for GitHub co-pilot and that includes any tool in your stack including your in-house tools keeping you in the flow across your entire day actually 75% of a developer's day is spent outside of coding Gathering requirements writing specifications and creating plans let's show how co-pilot can help with that live on stage for the first time so typically my day starts by looking at GitHub issues looks like we want to support a rich text input for a product description let's open workspace and get some help with that co-pilot interprets the intent of the issue to see what's required and it then looks across the entire code B base and it proposes what changes should be made this specification is fully editable and the whole process is iter but actually this looks pretty good co- pallet can now help us build a plan on how to implement this change all right that's a great start but we must not forget about our documentation so let's edit the plan and have co-pilot update our readme and then we can even get Coop palets help in starting to implement the code for us now this was just a simple example but in a large Enterprise codebase there are tens of thousands of files and dozens of stakeholders involved and that means meetings so many meetings workspace helps you focus on what you need to change and by the way as a developer I'm always in control I can see exactly what changes copilot is proposing and I can even get a live preview all right let's test out the input all right this looks great so I can go back and I can edit my code in BS code or I can submit these changes as a pull request to share with my team GitHub co-pilot co-pilot extensions and co-pilot workspace help you stay focused on solving problems and keeping you in the flow back to you sethia thank you so much ni I I tell you GitHub co-pilot and everything that that ecosystem is doing um is just bringing back a lot of fun and a lot of Joy back uh to coding and and really the thing about staying in that flow uh is I think what we all have dreamt for and dreamt about and it's coming back um that brings us to the very top of the stack Microsoft co-pilot we built co-pilots so that you have the ability to tap into world's knowledge as well as the knowledge inside of your organization and act on it now co-pilot has had a remarkable impact it's democratizing expertise across organizations uh it's having a real cascading effect right in fact it reminds me like of the very beginning of the PC era where work the work artifact and the work flow were all changing uh and it's just you know really having broad Enterprise business process impact it's lowering I I always say this it's lowering both the floor and raising the ceiling at the same time for anything any one of us uh can do since no two business processes are the same with copilot Studio you now can extend co-pilot to be able to customize it for your business processes and workflows today we are in introducing co-pilot connectors in co-pilot studio so you can ground co-pilot with data from across the graph from Power Platform fabric dataor as well as you now have all the third party connectors for SAS applications from Adobe atlassian service now Snowflake and many many more you now this makes the process of grounding co-pilot in first and third party line of business data just a wizard-like experience enabling you to quickly incorporate your own organizational uh knowledge and data we're also extending co-pilot Beyond a personal assistant to become a team assistant I'm thrilled today to announce team co-pilot you know you'll be able to invoke a team co-pilot wherever you collaborate in teams right it can be in teams it can be in Loop it can be in planner and many many many other places someone think about it right it can be your meeting facilitator uh when you're in teams creating agendas tracking time taking notes for you or a collaborator writing chats surfacing the most important information tracking action items addressing unresolved issues um and it can even be your project manager ensuring that every project that you're working on as a team is running smoothly these capabilities will all come to you all and be available in preview later this year and we're not stopping there uh with co-pilot Studio anyone can build Co Pilots that have agent capabilities and work on your behalf and independently and proactively orchestrate task for you you know simply provide your co-pilot a job description or choose from one of our pre-made templates and equip it with the necessary knowledge and actions and co-pilot will work in the background and act asynchronously uh for you right that's I think one of the key things that's going to really change in the next year where you're going to have co-pilots plus agents with this async Behavior you can delegate authority to co-pilots to automate long running business processes co-pilot can even ask for help when it encounters situations that it does not know much about and it can't handle and to show you all of this let's roll the video redefine business processes with co-pilot Studio create co-pilots that act as agents working independently for you simply describe what you want your co-pilot to do easily configure your co-pilot with the details it needs like instructions triggers knowledge and actions quickly test your co-pilot before you deploy and seamlessly publish across multiple channels watch it use memory for context reason over user input and manage long running tasks copilot can learn from feedback to improve and you're always in [Music] control put copilot to work for you with co-pilot Studio you know all around this stack is perhaps one of the most important things that we at Microsoft are doing which is wrapping it with robust security you know security underlies our approach with co-pilot co-pilot plus PC copilot stack we're committed to our secure future initiative uh you can see you'll see us make rapid progress across each of the P six pillars of SFI uh you know and the core design principles right which is secure by Design secure by default and secure operations uh you'll hear about throughout this conference in fact a lot more in Scott's keynote tomorrow how it underlies everything that we build and everything that we do so coming to the close I want to sort of you know there are many announcements that you will hear about at build but I want to go back to I think the core of what I think why we chose to be in this industry and why we come to work every day as developers which is the mission ultimately of empowering every person in every organization at the end of the day it's not about Innovation that is only useful for a few it's about really being able to empower that everyone and it comes down to you all as developers and Builders of this new world for us it's never never about celebrating tech for tech sake it's about celebrating ating what we can do with technology to create magical experiences that make a real difference in our countries in our companies in our communities already this new generation of AI is having an incredible impact thanks to all of you the passion you bring and the hard work you put in and I want to leave you with this one unbelievable example of how you're all building a more accessible World which means a lot to me uh using our platform and tools thank you all so very much enjoy the has to [Applause] build audio description is something that enables me to be able to watch a program or a film and get as much out of it as everybody else who is cited a white car drives down a road hands on a steering wheel I see art is a collective good I think everyone should be able to have access to Art audio description really helps me get the full experience a portrait of a group of 17th century Civic Guardsmen in Amsterdam the challenge though is that there are limited amounts of audio descriptions being Incorporated across media and entertainment Tech and AI have the potential to bring the Blind and low vision Community into the fold so at wpp we really care passionately about opening up access to content to people in a way that they want to consume it the tool that I've made is an application that allows you to upload videos and on the other end with gp4 with vision and as your AI Services you get your video back with spoken durations over the top kitchen scene with cat and Hellman's mayonnaise this makes audio descriptions cheaper and faster our goal is to be able to offer this product as a service for all of advertisement campaigns there are so many artworks in D Museum there are almost a million to describe ourselves it would have taken hundreds of years with AI we can do this in a matter of hours the subject is a male with a reddish beard and mustache visible brush Strokes that add texture and movement the first time I heard audio descriptions I it just brought me Delight it was this opportunity of oh my gosh I'm seen through the power of AI we're able to do things only dreamt about until recently when we strengthen our access to culture we strengthen the culture itself connecting our shared Humanity [Music] [Music] please welcome Executive Vice President experiences plus devices Rees ja good morning I'm rades you heard Satya talk about the copal stack the AI architecture the future now I want to talk about bringing that AI stack to Microsoft products first I'm going to focus on how we are expanding calet then Jeff teer will talk more about extensibility and then finally pan davui will share more on calet plus PCS and the windows ecosystem now diving into how copilot is evolving it was only an year ago at build that we showed you the promise of AI and then we made copal from Microsoft 365 generally available in November I want to spend a moment and talk about the journey that we've been on within Microsoft 365 we brought C- pilot to the applications that hundreds of millions of people use every day giving our users a powerful new way to interact with AI right in the flow of the their work now we've seen great adoption across a range of customers and industries nearly 60% of the Fortune 500 now use calet and we've seen accelerated adoption across Industries and geographies with companies like AMG and BP cognizant Moody NOA nardes Nvidia Tech Mahindra and many others purchasing over 10,000 seats each we've added over 150 copal capabilities since the start of just this year in addition we continue to integrate copal across more of our productivity apps and services from one note to stream to forms to one drive and more now we also launched a St alone copilot application whereas copilot had been embedded in Office Outlook and team the copal app now has all of Microsoft 365 embedded in it and what makes the copal app unique and uniquely powerful is the grounding it has first our copilot understands the web more than web grounding the copilot is crucially grounded in the users's work context who do they work with what do they work on their meetings their conversations the documents and that is the Microsoft graph the Microsoft graph represents the user their team their permissions their organization it represents their context now it's important to know that the CoPilot app is no different from any of other Commercial Services in terms of compliance and data handling Microsoft has no eyes on access and your data is not used to train the models so let's dive in and see how all this comes together in the copal app in the web tab you can get answers to simple or complex questions grounded in live web data all with commercial data protection now let's struggle over to the work tab here copilot is grounded in Microsoft graph meaning it has access to your personalized work environment the people you interact with important files teams meetings all your Communications by the way this is not a demo account this is my actual Microsoft account using real data to show you how personal this experience really is of course you're going to see some redactions I do want to keep my job now learning to ask the right question is key so you can get the most of the cop palot to help you there's a prompt library to give you suggestions just when you need them let me start by getting caught up with the latest my boss you may have heard of him go pilot understands organizational structure so he knows SAA is my boss it scans the latest emails and chats and files and in moments I have a detailed breakdown with updates that need my attention let's try one more here I'm asking copal to propose a session title based on this very keynote script which is a document and SharePoint and to create a list of topics on generative AI to discuss in this session copilot has analyzed a document for my work environment and propos a suitable headline but since copilot is also grounded in web data it is able to recommend topics sourced from the web for me to consider now Switching gears since its Inception calot has been a uniquely personal assistant as you just seen team but we all work in teams small and large organizations intimate and Global and we want to do more to go beyond to empower people when they come together and like Satya said today we are announcing team calot the expansion of calot Beyond a personal assistant this will enable copala to serve and act on behalf of a team a department and entire organization not just an individual user kopala will act as a valuable team member improving collaboration project management let's take a look team co-pilot expands co-pilot for Microsoft 365 from a personal assistant to a team assistant that becomes a valuable member of the team it gets added to a meeting and group chat like any other participant making it visible for everyone to interact with here co-pilot has been added to a me as the meeting facilitator helping the team focus on having a productive conversation co-pilot shares the agenda in the meeting chat and anyone can add topics without interrupting the active discussion it actively tracks time providing reminders when it's time to move on to the next topic co-pilot will also take notes during the discussion that anyone can edit or add to and flags followup tasks for everyone to see let's switch over to teams chat here co-pilot has been added as a collaborator to help streamline Communications as the conversation happens co-pilot maintains an upto-date view of important information for everyone to see this includes surfacing an unresolved issue for the team co-pilot will also help the team work together to resolve open issues here a team member asks co-pilot for recommendations and grounds The Prompt with a relevant document co-pilot instantly responds with suggestions for the team to consider once once the team has agreed on the solution co-pilot automatically updates the notes pain removing the open issue and detailing the decision made with Team co-pilot assisting in new ways the entire team will be more productive and collaborative the team kobalad will be a valuable new member of any team and these initial capabilities will be available to our c customers in preview later this year now in my conversations with the customers one of the top questions I get is how do I translate productivity gains into transformative business results and simply put the answer is moving to a reimagining of business processes using your own copilot and agents and extending Microsoft copilot now Jeff is going to walk you through our extensibility platform is going to enable you to do just that but first let's take a look at the copal architecture that really makes all of this possible the copal as we spoke can recall and reason over upto-date web knowledge it is grounded in search all copal experiences have this web scale now in addition to being grounded in the web it is also grounded in your data through the Microsoft graph and when the copilot is in an application like office or teams or Edge it also understands the application context so for example in PowerPoint it is able to draft a slide deck for you from a document the Microsoft copilot is architected to compose or inherit capabilities based on the user context and now as a developer you can build copal extensions at the data layer at the experience layer to further extend and customize the calet and all of this is Enterprise grade with tools for it to manage and personalize for employees a leading isvs are already working with us Building Solutions that can extend the Microsoft calot and I would like to share two examples from service now and Adobe for service now as you know service now helps organizations orchestrate Anatomy tasks and processes across their Enterprise here we see service now's copilot extension now assist responding to user prompts inside copilot from Microsoft 365 with the exact same knowledge functionality and user experience as it has today in teams service now has included several custom zero query default promts to help users get started with the most common tasks without having to know how to craft the right text to start the conversation over to Adobe who are working on bring bringing bringing Adobe experience cloud workflows and insights to Microsoft 365 and copilot with Adobe Express copilot extension users stay in the flow of their work in a Word document and can start a workflow in Adobe Express that allows them to create social content select and added images and Stage it for publishing so as we close out this first chapter I hope you're excited Microsoft copal is already helping people people save time be more productive and creative team copilot experience copilot in meaningful ways and then there are great developer opportunities for you to extend Microsoft calet Jeff teper will join us now to share how easily you can build copal extensions but first I want to close my section with a video showcasing how Lumen is using copal for personal productivity to enhance their sales processes and connecting copilot to their systems using copilot connectors let's roll the video Lumen is a company that's going through a tremendous transformation an evolution to move from telecommunications to technology our goal is to help our 3,000 Sellers and customer success professionals shift away from transactional selling and move to being customer obsessed co-pilot for Microsoft 365 helps us move faster it's really hard for a seller to learn everything about all their customers and one way they can do this is using co-pilot for sales what Tak sellers hours they can now do in minutes co-pilot and Salesforce for us that integration has been vital people are coming to the table more prepared they have more information they're armed they're ready to have customer conversations before co-pilot with these third party data connectors the seller would have to go out to all disparate systems Salesforce gain site service now co-pilot connectors help to solve for the fragmentation that customers can feel which drives customer satisfaction we have seen an increase in outbound calls by 40% customer Outreach typically takes 4 hours now with Microsoft co-pilot with all of these third-party data connectors that is a 15minute journey if we can give our sellers back 4 hours a week it's worth $50 million in annual revenue and that is math that matters when you unlock people teams and culture you unlock growth we're at an intersection it's time for disruption in this industry and we are here for it well as Rees shared you will able be able to easily and securely use your applications and knowledge to build co-pilots that help your employees and organization be far more productive and grow your business and you can now extend the Microsoft co-pilot with your own co-pilot with handoffs in all the co-pilot experiences as well as in Microsoft teams where you can reach hundreds of millions of users today for both the personal and group assistance scenarios that reesh outlined and we are making building these co-pilots even easier from a few clicks in SharePoint to more advanced customization in co-pilot Studio to full control of your models your data your applications your actions your experience in Visual Studio code so let's first look at what this means for end users co-pilot extensions run everywhere that co-pilot is the Standalone experience across Microsoft teams and as we're showing here in the Microsoft 365 app on the right you can easily browse your installed co-pilot extensions find new ones or build your own which I'll show coming up and how Microsoft co-pilot works is it reasons over the user's prompt and Maps it to the right extensions or you can explicitly at mention that extension like we're showing here you're going to be able to drill into a deeper focused conversation with the extension like we're doing in this marketing example that has suggested prompts for quick actions and to just show the users the capability of your co-pilot this allows the Microsoft co-pilot to have realtime access to knowledge and applications in your environment here we're using a suggested prompt to ask about a key feature of a delivery drone the co-pilot extension you build is going to come back with a visual adaptive card bringing in all the information to avoid an unnecessary multi- turn conversation so the user can just focus on getting their work done and again these co-pilot extensions also run in teams in oneon-one and group chats and channels and in meetings so you can reach all these users today all right this is build so let's get to building with our first custom co-pilot extension you're going to be able to do this with uh from again a few clicks in SharePoint to Advanced customization co-pilot Studio to visual studio code we'll start in SharePoint which is often the authoritative source of knowledge and content processes with Advanced collaboration workflow and security all of which co-pilot honors to make sure users only get access to information uh that they're they have permission to I'm in this SharePoint site I'm going to go ahead and select a few documents hit click to co-pilot and right there that looks good I'll go ahead and change the name of this the the delivery drone that looks fine and just like that I've created my first custom co-pilot that you can use to extend the Microsoft co-pilot uh let's go ahead and try this one out how much does a delivery drone service cost and we can see it comes back with a flat fee of $5 per order all that looks pretty good uh secure grounded we're going to go ahead and share that with our team and up comes the standard sharing dialogue to honor the security in your organization we're going to copy this link go into teams paste it in the chat we'll say try this out we will go ahead and paste that and just like that in seconds we've created a secure grounded custom co-pilot and shared it with our team and uh Microsoft teams so we're very excited about letting anybody create create these secure custom co-pilots and this support will be available in SharePoint this summer sign up today for the preview we're very excited about that all right next yeah next on the Spectrum we're going to do some more advanced customization in co-pilot Studio starting right from SharePoint I can launch into co-pilot studio for my more advanced edits and you can see all the information for the co-pilot I just created is carried forward uh so that's all there from SharePoint and let me test this out by asking when the launch event is and it'll return back that it's on June 20th but what I really want is that co-pilot to do work for me not just answer questions and so for this I need to go into co-pilot studio and start by adding additional data sources in the knowledge tab here we can add websites and files and connect to over a thousand co-pilot connectors uh in this case our account information is in two tables in data verse and so I'll go ahead and select them and we've got the information we need next we move to the actions tab where I need to see if somebody's already registered for the event and if not send them a personalized invitation our event registration is managed in an external system so we've created a custom connector to go get it and you can C us configure that we can tailor the action and input and outputs however we need and then the second step is to automate the sending of that personalized invitation we've built a custom power automate flow so we'll add that as a co-pilot action as well and there you've go in pretty quickly we've got a complex co-pilot extension with content from SharePoint data from dataverse two disparate actions and we can go ahead and test this in co-pilot Studio to see if Koso is actually attending the launch event it says they're not and the co-pilot conversation helps you see why what's happening and map that all through we can ask a follow-up question again that's routed to dataverse about who the account manager is and we can see that's Perry Lang and then we can ask co-pilot to go ahead and send the invitation now again what's happened here is the conversation was entirely generated for me identifying and chaining together the key Knowledge from data divorce and the appropriate actions we added earlier with generative AI capability last we're ready to publish this co-pilot extension back to SharePoint Microsoft co-pilot and teams and from teams you can search for it in the unified Marketplace I can add it to one-on-one or group chats or meetings Etc and again I can see this in the microft co-pilot experience that same Relic Cloud co-pilot and continue the exper experience there so we're very excited this is a pattern that people are already doing today let me show you what one of our customers Walters cluer who's a leader in information and software and solutions is doing to enhance their tax and accounting professionals workflow with co-pilot they're building a co-pilot extension to enable their accountants to complete each step of their workflows interacting with their backend system just using natural language with no context switching co-pilot takes actions on their behalf saving time and cutting the process down from minutes to seconds and it's time to communicate back to their client the extensions helps close the loop drafts an email attach the estimates and ensures a seamless end to-end productive experience so we're excited about all of you building these kinds of custom co-pilots and co-pilot extensions co-pilot studio is now generally available to build Enterprise grade co-pilots the new capability to publish co-pilot extensions from your co-pilot is in private preview and as you heard SAA say earlier co-pilot connectors are now in public preview and they make it even easier to connect your co-pilots to your business data your apps and workflows okay last we want to show you how to build a co-pilot extension as a professional developer with full control of your models your data your actions your experience in Visual Studio code so let me go ahead and do that so what we've got here is Visual Studio code with the teams AI toolkit installed and we've loaded up a template for building a custom co-pilot and using it as an extension uh the first thing you see in the code is where I configure the model uh you can use an off-the-shelf model here we're using one from open AI you can use a refined model or you can build your own completely tailored to what you need if we go scroll down the code a little bit farther this is where rag integration is and you can see in a few lines of code we've integrated our data in this case azure's uh Vector search capabilities you can use any other data source uh the Azure AI search capability is a great one to use uh we keep scrolling down we see the actions uh uh defined uh and registered but here is where we go search for our product inventory and you can see uh We've inserted a break point so that we can follow this along because I want to show how easy it is to do end to end development here so we'll bring up teams where we've installed that co-pilot extension for the inventory find information for the chai te product hit return and we're paused why because we've hit that break point and so we can go back into Visual Studio you can see sure enough it's fired and if I hover over the parameters you can see the product name chai has been passed to it so this is incredibly powerful right from within Visual Studio to do end to-end debugging across the Microsoft apps co-pilot and your co-pilot extension if we keep going down before we return that to the user I just wanted to show what the ux looks like here's an Adaptive card that comes back it's defined in Json but you can also see the user experience for that that all looks good let's go ahead and resume execution from the debugger you can see that we've now returned uh back into teams uh the result and we've got an attractive adaptive card that gets the user the information they need does handoff with the co-pilot so right in line the user can complete the work so pretty exciting Again full flexibility within the team's AI library and uh Visual Studio code we're excited just like with co-pilot Studio this is something you can do today targeting hundreds of millions of teams users and very soon the ability to turn your custom co-pilot in an extension will be available uh and this is not new this is something that hundreds of isvs are already doing today uh across all sorts of experiences in Microsoft 365 teams and now co-pilot just two examples are leading software organizations ezri and Thompson Reuters ezri is the market leader in geographic information systems they're building a co-pilot extension that adds spatial analytical capabilities directly into teams meetings so that users can ask co-pilot from Microsoft 365 a question about map data and co-pilot will seamlessly hand off to a rich interactive experience with visualization in EZ's custom co-pilot with all the associated context next up is Thompson Reuters who's a lead leading Global in content and technology company that is transforming the legal profession with AI Thompson Reuters is extending the co-pilot experience in Outlook and word in teams for things like risk assessment so that based on the content of the Outlook email they can update the policy documents in word and communicate those policy changes to reduce risk right within the teams meeting we're super excited to see what you do targeting this huge user base and we're going to help promote and distribute your application through our unified Marketplace again reaching hundreds of millions of users today in Microsoft 365 and in teams and one of the reasons this Marketplace is trusted is it has confidence in the full governance capabilities for their own applications and custom extensions they build as well as the ones they will get from all of you building them in a vibrant third- party ecosystem so to recap we have a simple powerful platform for AI and Microsoft 365 that you can use to be far more productive across the full spectrum from something everyone can do in SharePoint to Advan customization and co-pilot Studio to the full power of visual studio and visual studio code but wait there's one more thing we are very excited to make teams a fantastic place for developers to work work together with AI to write better code faster and so we've got a whole set of announcements around that this week at build as well First Things First Source Code inside teams with syntax formatting and get this we wondered what would get the Applause and that was my bet and but wait you should have held the Applause with Microsoft Loop Co editing of that source code right within teams but yes but there's more developers have asked us for a while for greater information density in teams so you can create and switch to compact modood Mo compact mode to see much more content on the screen and you can be much more productive with things like keyboard shortcuts and new slash commands for teams and of course developers always in the flow of resolving issues in chat and one of the features we're really excited to announce is meet now so that right within chat you can bring up a ringless call between members of the team and resolve the issue in seconds and last developer teams love to have fun to break from the stress so you can use custom emojis reactions now in Microsoft teams and of course this is building on top of a growing set of Partnerships with devop tools jira dater dog pager Duty uh and of course deeper into integration with GitHub and much more integrated with Microsoft teams so we are very excited to make teams a great place for developers to work together to build this next Generation AI the next chapter is how Windows is the best platform for building that next Generation Ai and to show that I'm excited to invite pavin to the stage pavin [Applause] thank you Jeff good morning it is great to be here at build this is one of my favorite times of the year connecting with fellow product makers about the world's canvas for Innovation Windows I'm excited about the copilot extensibility that mess and Jeff just shared it really Shines on Windows the platform customers choose for Microsoft 365 and co-pilot over the last year we've learned so much about how co-pilot can best serve you we're working hard to make it even more valuable with a vision for co-pilot meeting you right in your workflow imagine creating a presentation from a document in file explorer we're helping customers troubleshoot their PCS using quick actions and natural language right in settings we're focused on making copilot even more contextual and useful across windows we took a big big step towards that goal yesterday with the announcement of copilot plus PCS the fastest and most intelligent PCS ever built AI is woven into every layer of these devices from the Silicon to the operating system with the most powerful PC npus capable of delivering over 40 trillion operations per second this new class of PCS is up to 20 times as powerful and 100 times as efficient for running AI workloads compared to traditional PCS from just a few years ago built together with our silicon Partners AMD Intel and Qualcomm and our OEM Partners these PCS will be available June 18th starting with Qualcomm Snapdragon X series of chips copile plus PCS are redefining what you can do on a PC and setting the direction for the next decade of Windows to put this new wave of AI inovation in your hands we're excited that Qualcomm has announced Snapdragon devkit for Windows it is designed to be your everyday Dev box for AI with the power and flexibility you need as we Define this new path for Windows in the era of AI one thing that will never change Is our commitment to openness we recognize that the real value of Windows comes from the energy and the innovation of the ecosystem it comes from from all of you so as we enter this new era let's talk about how we're going to serve over a billion Windows customers together as SAA said earlier building a powerful AI platform takes more than a chip or a model it takes reimagining the entire system from top to bottom the new windows co-pilot runtime is the system that extends the co-pilot stack to Windows the windows co-pilot runtime is a new integral part of windows 11 and has everything you need to build great AI experiences whether you're just getting started or already have models of your own it includes the windows co-pilot Library a set of apis that are powered by on device models that ship with Windows and includes AI Frameworks and Tool chains to help you with your own on device models and it's built on the foundation of powerful client silicon including the npus in the co-pilot Plus devices let's take a look at how the windows co-pilot runtime enables an entirely new class of experiences OS experiences like recall that help users find anything they've seen on their PC inbox app experiences in photos and paint which let you bring your ideas to life using realtime image generation and app experiences like cap cut and seable and D Vinci resolve some of our first Partners using the new npu and helping us build the windows co-pilot runtime looking ahead the Xbox team has a vision for using the windows co-pilot runtime to empower players and game developers let's take a look [Music] [Music] that's pretty inspiring this entire class of new experiences now benefit from faster task completion enhanced privacy and lower costs by using the windows co-pilot runtime next let's take a look at the windows copilot Library the apis and models that support them let's take the recall experience as an example it relies on on device models deeply integrated into Windows to capture context on the screen that data is transformed into vector embeddings and indexed in a vector store called the windows semantic index the recall user activity API allows you to extend your app into to recall so users can jump right back to where they were in your app and increase your app engagement in the same way Edge and Microsoft 365 apps like Outlook PowerPoint and teams already have in fact soon recall will draw context from the Microsoft 365 graph to build your own semantic index store you can use the vector embeddings API that makes it possible to use retrieval augat generation or rag Within your applications with your data imagine you have Wind forms or WPF app that works against a large Corpus of sensitive data with Vector embeddings API you'll be able to create on device Vector stores for those records that's powerful when combined with the rag API to enable natural language search in your applications for your users of course that's just one example the apis in Windows co-pilot library cover the Spectrum from low code apis to sophisticated pipelines to fully multimodal models like the recently released 53 the single best slm in the world 53 mini does a better job than models twice its size on key benchmarks today we're thrilled to announce F silica built from the F series of models specifically designed for the npus and copilot plus PCS it offers lightning fast on device inferencing and state-of-the-art first token responsiveness Windows is the first platform to have a state-of-the-art slm custom build for the npu shipping in box now let's take a look at what you can do to bring your own on device models to Windows using Frameworks and Tool chains it starts with direct ml the lowest level machine learning framework in Windows similar to direct X for graphics whether it's your own open- source models or an open first model from hugging phas direct ml helps you scale the breadth of your efforts across the windows ecosystem by giving you to the metal access to gpus and npus we also know that a lot of you do your development on pytorch on Windows and we're thrilled to announce that windows will natively support pytorch through direct ml that's right pretty exciting native pytorch support of course means that hugging face model will just work on Windows and not just that we're collaborating with Nvidia to bring these workflows to over 100 million RTX AI gpus in the windows ecosystem now that's incredible you can download the pytorch and directl developer preview today we're also going to extend direct ml to our web developers by introducing webinn on Windows webinn is a web native machine learning framework Microsoft has been working with Intel and other Partners to unlock the access to local ml accelerators so you can build performant AI experiences in your web apps behind me you see clip Champ's autoc compose feature achieving faster video composition experiences and Cloud savings by leveraging the npu through ort web and webinn I'm excited to announce that webinn is available in developer preview today okay so that's a glimpse of the windows copilot runtime how it lays a foundation for Innovation giving you the largest catalog of models on the largest ecosystem of devices making windows the most open platform for AI I heard you there that's fantastic as Windows transforms for the a of AI we're continuing to to reach the expanse of the platform including all the AI experiences you create with the windows co-pilot runtime we're delivering windows from the cloud with Windows 365 so your apps can reach any device anywhere and we're introducing Windows experiences to new form factors beyond the PC for example we're deepening our partnership with meta to make Windows a first class experience on Quest devices and windows can take advantage of quest's unique capabilities to extend Windows apps into 3D space we call these volumetric apps let's take a look workflows are transforming with mixed reality Microsoft is partnering with meta to bring Windows 365 and local PC connectivity to Quest and enable developers to easily extend their Windows apps into the 3D space PTC has been working with this platform bringing Creo into mixed reality in under a day this extension allows users to enhance spatial understanding without leaving the app that powers their work sign up for the developer preview today as developers you'll have access that's great to hear um as developers you'll have access to volumetric API and this is just one of many ways to broaden your reach through the windows ecosystem for decades Windows has been a stage for the world's Innovation with pilot plus PCS the windows co-pilot runtime and Windows 365 we're going to unlock a new era of innovation together thank you back to you Rees thank you pan and thank you Jeff now we've covered a lot of ground over the last 40 minutes from the expansion of copilot Beyond a personal assistant to acting as a valuable team member to how you as developers can extend Microsoft copilot with your own copilot and agents in just a few clicks in SharePoint to more advanced customization in copilot Studio you can use vs Code full control of your models your data and actions and of course a phenomenal opportunity for developers with over a billion Windows customers so I'm going to close with highlighting another customer Amgen a pioneering biotechnology company that harnesses the power of biology and Technology to fight the world's toughest diseases we are going to see how they harness Microsoft copilot in their mission Kevin Scott our CTO along with some special guests will round out the day one of Keynotes but before the video let me just finish by simply saying thank you thank you for spending your time with us here at Bill it means a great deal to all of us and thank you for the trust you play working together building the future with us every single day let's roll the video and thank you we are at a once in a generation moment where we have the opportunity to harness the power of tech and biotech to revolutionize drug Discovery and development manufacturing and commercialization we started with 300 licenses and now 20,000 of our employees have access to co-pilot in manufacturing your moving at a very fast pace you have a lot of data coming at you co-pilot is one tool that allows us to synthesize that information and ensures that we can focus on what's most important the production of that drug we've taken advantage of extensibility options using graph connectors identifying opportunities to use co-pilot for daily efficiency gains and multiplying those by a 100 or a thousand times in the life of a molecule from idea to achieving marketing authorization starts to give you a sense for what's [Music] possible we have capabilities in our hands with these new tools in the early days of this new platform to Absolut abolutely do amazing things where literally the challenge for you all is to go do some legendary that someone will be in awe of you for one day when people are looking for food resources it can be a humbling experience childhood hunger and food insecurity is a problem that we can solve and we are using AI to connect those in need we know through research that people prefer sometimes to talk to an AI agent talk to a chatbot because they're not talking to a human technology can really help people get what they need with multiple resources in one place and it's making it mobile friendly which is really important most people access this information via smartphone what we're doing at no kid hungry is ensuring that we end childhood hunger around 2019 Brian started having some issues with his ankle I was able to get him an appointment with a doctor and he was like I hate to tell you but this is ALS how would I live in a world where my kids wouldn't be able to hear me say I love you the biggest opportunities when it comes to emerging Technologies is the gap that I can fill AI has the ability to increase communication increase Independence it's the next wave for accessib what Microsoft is doing their voice banking technology really gives someone back that intonation and a little bit more of their personality today we're going to play like a newer version of your voice so you just choose any phrase and then I'll play it on here as well hey beautiful people I am so happy to be with all of you all right I'm going type it out hey beautiful people I'm so happy to be with all of you sing I think this sounds just just like me and I am overwhelmed with joy that my family gets to experience this part of me in many ways I feel like I am [Music] back please welcome Chief technology officer and Executive Vice President of AI Kevin Scott [Music] [Applause] thank you all so much for being here with us today so I've been fascinated my entire life by tools and the power that they give us as individuals and teams to really create extraordinary things um in my own personal making my wife sometimes wishes that the ratio of tools to extraordinary things was a little bit different than they are um but as a person who has dedicated their entire career to Building Systems and infrastructure tools and Frameworks for other developers to use to go make the things that they need to make for whatever reason that they need to make them it's just extraordinarily gratifying to see the really truly consequential things that you all are choosing to do with these new AI tools that we had a rooll in bringing into existence so I just want to thank all of you so much for all of the great that you have made over the past year so you know I want to give a call out to our friends at uh share uh our strength um you know but it's not just the really wonderful work that uh we just showed in the video that folks are doing like as Saia mentioned earlier uh we have over 50,000 customers using Azure AI tools and the Azure AI platform um and like this range of customers spans everything imaginable from all different industry categories all different scales of business from small startups to Fortune 100 companies and all different stages of exploration of how to use AI to do transformative things from doing Explorations like trying to find product Market fit to scaling things where you found the product Market fit to folks who are just trying to figure out how to optimize and enhance the things that they've been doing for a very long while so I just wanted to highlight another couple of things uh that I thought have been really noteworthy uh collaborations that my team has had with some of you all in the audience um so we've been doing some really cool work with Etsy uh using generative Ai and the Azure AI platform to build features like their new gift mode uh so we have have the CTO of etsy here in the audience with us today [Music] hello and you all should go check out gift mode like it is uh like a really Innovative way to help you figure out how to buy things for people who are difficult to buy for uh which I'm guessing like almost all of us in the room are it's like super stressful trying to figure out what the right gift is for the right person that you care about in your life U and this is just one of many interesting things that Etsy is doing with uh generative AI to really enhance the experience of the Etsy product and to bring Delight to their customers um another thing that I'm super excited about is uh the work that cognition has been doing so uh Scott uh and the cognition team are also here with us today um we have just recently announced a partnership between Microsoft and cognition their product Devon is like an absolutely amazing uh tool um so like if you can imagine for yourself like some of the most tedious things that you uh that you do as an engineer or software developer Devon is a tool designed to help you with those tasks like I can't even tell you all the number of times as an engineering leader I've either or as an engineer that I've had to uh write code or lead teams writing code for doing things like rep platforming uh and application and like re-platforming is like one of the laws of physics of of like how we build software systems like it's a thing that must be done but like you know it's rare that an engineer really enjoys doing that and so like the incredible work that Devon is doing on top of these incredibly powerful tools uh is just really extraordinary and we're super excited to be partnering with them and to bring all of the Power of what they're doing to Azure and having their systems and infrastructure run on Azure so you know I I really want to talk with you all today about just a couple of simple things um what's driving all of this progress like why is all of this happening right now and so part of it is like we're riding an extraordinary platform wave like something is fundamentally changing in the universe of Technology much in the same way that it changed when we were going through the PC Revolution where Mo's law was driving uh an incredible uh increase in the power and lowering of the cost of personal Computing which led to it becoming ubiquitous and you know something that we now all get to take for granted similar thing happened with the internet Revolution where networking technology connected all of this compute together and allowed us to do things that previously were unimaginable and we're going through one of those major technological changes right now uh being partly driven uh by a set set of things that we'll talk about in a few minutes uh just the incredible scaling of the capability of AI systems as you apply more compute and more data to training them but like before we get to that expansion of the frontier of the increase and those capabilities like a super important part of the emergence of a new powerful platform is sort of completing the stack so it's actually hard work uh even when you have a piece of technology that is improving at an exponential rate to figure out how to do all of the things that have to be done in order to deploy it in real applications so that you can go out and deliver value to real customers who care about what it is that you're doing and we've done a huge amount of work over the past year on the co-pilot stack um it is both optimizing a bunch of systems so things are getting cheaper and more capable and it's also building that whole cloud of capabilities and systems services and Tools around the core uh AI platforms the big models that you all need uh and the choices that you all want so that you can build the things that matter to you under the constraints that you're operating under so one of the reasons that we have been able to do this is no other company has deployed more generative AI applications over the past year that Microsoft has and so you have probably heard us over the past year talking about all of these different co-pilots like this new software pattern that we originated with GitHub co-pilot where you pair powerful generative AI uh with this user interface Paradigm where you're using the AI to help assist users with task and so you can apply this to everything and I know many of you in the audience are building your own co-pilot so Microsoft itself is building co-pilots for service for sales a co-pilot in bing co-pilot in Edge co-pilot in Windows and the reason that we've been able to do all of this work is because we have the co-pilot stack that we built for ourselves to help us have real agility in getting these products built quickly um to have them built efficiently where their price and cost optimized uh and to build them in a way where they're safe and secure um one of the things that you have heard from Rees and that you'll be hearing a lot more of it build is like part of what the co-pilot stack is allowing us to do is to unify The Experience across all of these co-pilots into one logical Microsoft co-pilot where you don't have to really pay attention to which Microsoft product or service you're in like the co-pilot just understands all of your context and delivers all of the capability of the model in the context of your data and your task to you when you need it so the other thing that is really driving progress is not just this sort of completion of the co-pilot stack um this sort of progress that we're making in filling out that toolkit for you all so that it is easier for you to build software but we are riding like a fundamental wave in um in the development of this AI platform where if you just sort of look at compute over time like how much uh how much GPU Cycles or accelerator cycles that we're using to train the very biggest models in the world since about 2012 uh like that rate of increase in compute when applied to training has been increasing exponentially and we are nowhere near the point of diminishing marginal Returns on how powerful we can make AI models as we increase the scale of compute so we're sort of doing two things at once uh at Microsoft so we are optimizing the current Frontier uh and like building that toolkit to help you all leverage it while at the same time investing at a pretty incredible rate in pushing the frontier forward and so one of the super interesting things that has just happened uh as we're sort of pushing the uh the frontier forward and like really really driving forward on efficiency is what our partners open AI launched last week uh in the form of GPT 40 as SAA mentioned earlier uh GPD 40 is like a really stunning achievement uh multimodal model that understands uh a bunch of different input types from video to text uh to speech uh that can respond in a bunch of Rich ways from uh text to speech uh and eventually video uh it can respond to you or to the interactions uh for users in their applications in real time uh and in the case of the chat GPT demos that folks have uh seen you can even interrupt the model uh so that you can have really fluid interactions with these systems and then an enormous amount of work has gone into GPT 4 both the model itself as as well as the supporting infrastructure around it to ensure that it's safe by Design so I wanted to also just remind folks uh like this efficiency point is is real so while we're off you know building bigger supercomputers to get the next big models out and to deliver more and more capability to you like we're also grinding away on making the current generation of models much much more efficient so between the launch of of gp4 which is not quite a year and a half ago now it's 12 times cheaper to make a call to GPD 40 than the original chat uh or the gbd4 model uh and it's also six times faster in terms of like time to First token response so it's just really extraordinary like how much progress we're making because of like the full set of optimizations that we're doing from the Silicon that we're building networks data center optimization as well as like an incredible amount of software work that we're doing on top of all of this hardware and infrastructure to really tune the performance of these systems and the great thing is is again there's no point of diminishing marginal return here and one of the messages that I want to land with you all today uh is that you can count on things getting more robust and cheaper uh at a pretty aggressive clip over time so it's like a really important thing to internalize like something uh you know like we challenge ourselves on at Microsoft all the time is like aim for things that are like really truly ambitious because like all of this optimization work is going to acue to make things uh like really ubiquitous in terms of like how you can go deploy them um I just want to show a little quick demo video here uh so let's roll the video show you hey Microsoft build conference is tomorrow and I have not been sleeping and I am making stupid mistakes can you help me take a look at this code I'd be happy to help could you tell me a bit more about the issue you're facing with the code for example are you getting an error message or is the code not doing what you expect it to do let me just show this to you it looks like you're working on a python function to summarize main points from a keynote what I'm trying to do do is go through a list of keyter and summarize all of their work and my problem is that I wanted to just add Kevin's name to the end of the keynote list and it added every character of his name instead of just his whole name what did I do wrong here change keynote list. extend to keynot list. append last keynote this will add Kevin as a single element to your list just like Saia and rajes O that sounds good let me try that sounds good give it a go and let me know how it works okay I replaced extend with a pench does that look right yes that looks better all right I made that change let me run it and try sounds good let's see how it goes it worked SAA rajes and Kevin all came out awesome I'm glad it worked out sometimes the smallest things can drip us up especially when we're sleep deprived thank you so yeah I mean it it really is extraordinary so like I should say by the way that Jennifer would never make that actual mistake uh in writing a python application but Kevin might uh so like this is all giving me hope that a rusty old CTO might one day uh be able to program again if uh if the need arose but but like I I do want to make sure that we're paying attention to like just how much has changed over the past uh year I mean like what what you just saw would have been absolutely inconceivable to think about actually working and like this was you know not a tortured demo like you know we were sort of looking for something to show off gbd4 to you all uh last night uh and Jennifer showed me this and then she just recorded this demo like this is uh just crazy that it works this well um so another set of things that have really making a huge amount of progress is uh like what's possible with smaller models um so we have been working for a while on this series of models called fi that uh are small language models so Saia chatted a little bit about this in his keynote earlier um and the way that we think about fi is uh just imagine an efficient Frontier and so usually when you're building these models uh you're trading uh a couple of things off so you can trade size off which is related to you know performance and cost and a whole bunch of other things versus quality so the smaller the model is like the cheaper it is to do inference and like the less compute that you need to actually run the model uh so small models are more amable to running on devices but it usually means that you have to take a hit on quality and like what we're discovering uh in particular over the past year is that there's this notion of an efficient Frontier so we don't even show the GPD 4 o point on this slide it would be like way way way off to the right uh just in terms of the size so like if you want extreme levels of quality and performance like a frontier model is your friend but in some cases like you may want to choose one of these other models somewhere else on this efficient Frontier where the trade-off that you're making between uh cost to serve or latency or locality uh is acceptable given the quality you can get and the very interesting thing that's been happening over the past year is the quality that you're able to achieve in these small models is getting pretty high um so I just wanted to show this as an illustration so like you know remember back uh ancient history the to the launch of chat GPT in November of 2022 so chat GPT launched on top of gbd 35 uh and so like everybody was just absolutely Gob smacked uh at what was possible with with GPT 35 I mean just a sort of a stunning revolutionary thing that happened we fast forward uh a few months uh to March 2023 and GPT Chad GPT gets an upgrade to gp4 which is even more extraordinary uh what it's able to do uh like you're able to ask extremely complicated questions of these things and get very rich uh interesting compelling completions so fast forward to today and like you can sort of see that a version of 53 optimized to run on a mobile phone can respond to a prompt just like chat GPT could uh just a year or so ago uh with responses that are sort of equivalent and like this is not arguing that you know 53 that's running on this device is just as powerful as gbd4 it is not but the way that you all should be thinking about it is in many cases the models can be appropriate to use for building your applications uh when you have a particular set of constraints that you're trying to optimize towards and so like all of this is kind of you know abstract in a way and so I I wanted to really motivate why this matters uh with the following example um so Saia mentioned earlier the partnership that Microsoft has formed with KH Academy and KH Academy's Miss is really interesting and important like they are trying to ensure that every learner on the planet no matter where they are has access to high quality individualized instruction and so one of the things that we are exploring together with KH Academy is the possibility of achieving that goal of ubiquity of these personalized learning agents uh by using something like 53 where you can imagine training a 53 model that's very good at something like math instruction uh so this is an actual interaction with 53 medium that has been fine-tuned to work particularly well for math tutoring um and the challenge with doing something like this is that you have to not just have the model give the student an answer but like you want it to lead them towards discovering the answer themselves so like a tutor is very from an answer agent and so like it's just exciting to think about how many tools that organizations like KH Academy have to solve like these really really important missions that they have in the world and so with that I'd love to bring uh Sal KH uh from KH Academy onto the [Applause] stage hey s thank you so much for being here with us today so um you know we have been chatting about your mission I think for quite a while um and you know one of the interesting things that happened when chat GPT burst onto the scene a few years ago is that there was this reaction from a bunch of Educators and like I think it was actually a reasonable reaction where like okay we don't understand this like we don't want our students using it they're going to like do things that we would prefer that they didn't do it you on the other hand looked at this and said this is amazing and leaned all the way in can you explain a little bit about what drove your first reaction reaction to this new technology yeah you know some of you all know how KH Academy got started if you go almost 20 years ago it started with me tutoring a cousin I was a hedge fun Analyst at the time I tutor one cousin word spreads in my family free tutoring is going on before I know it I'm tutoring 1015 cousins and I start writing tools for them software I started making videos that's what a lot of people are know about KH Academy and if you think about that Journey from then till now even right before we started really working on generative AI everything we've been doing is how could you scale that type of personalization that I was originally doing with my cousin naadia and we were approximating it with software and videos and teacher tools but when we saw but to some degree there was going to be we're going to Asm toote on how far you could get with pregenerative AI tools and then when we saw and it was really gp4 that opened our mind uh you know Greg and Sam from open AI showed it to us end of the summer 2022 and we realized there's things that had to be worked out but it could get that much closer to emulating what a what a real tutor would do and it was obvious it could also be used as a cheating tool and you have to worry about safety and privacy especially with under 18 users but I told the team let's turn those into features let's put the guard rails on it because this could get us that much closer to our mission which is free worldclass education yeah I think one of the other things that you all have done uh and like this is a really important thing to internalize about these models and systems is like the model isn't a product and like the systems aren't silver bullets like you still actually have to understand like who your customer is like what problem you're trying to solve and like how to go deal with a whole bunch of gnarly things on top of the you know know this incredibly interesting and Powerful tool so you can do something useful like you want to talk a little bit about what you had to do there yeah and I have to admit and maybe everyone a lot of people in this room or in the world right now are experiencing this every now and then you see some of these demos and you're like does my application even have relevance anymore this thing is going to be able to do everything but then when you sit down and you really think about how a school system a teacher a stu a student's going to use it and you're going to what are the guard rails what are the Privacy how do you make sure that it really does the tutoring interactions appropriately it's to standards you realize that there's a lot to do at the at the application layer now I think we're all discovering together this new world of developing applications on terms of on top of large language models it's not deterministic in the traditional way you have to have evals you have to constantly test it uh but but we're we're realizing that there's just so much to do it really is a bit of a it's a very exciting time yeah I mean one of the things that I'm especially excited about is like this mission that you all have for ubiquity and like the partnership that we're doing with you all is going to in aable you to get the hand or like get every teacher in the United States hands on kigo and your tools and you know like just a personal anecdote for me is like my daughter is in the ninth grade she's taken biochemistry and just in love with science in general um and she on her own without any prompting from Dad uh figured out how to use the free version of chat GPT to take a bunch of biocham papers that were like way way way more complicated than a 15-year-old by rights has to understand dump them into chat GPT and then just ask a million questions about it and like her learning acceleration because she's figured out how to use this tool is extraordinary and like I just want every kid in the world to like have the same experience that my daughter has absolutely and you know what we realize at Khan Academy there is a subset of students that if you give them the tool and it sounds like you're lucky enough to have a daughter like that they will run with it but what you really need in most cases is you need caring adults primarily teachers in the room most motivating students driving that usage and so what we're really excited about this partnership and this this is a big deal I want to make sure you know we are using state-of-the-art models that use real compute it has real cost associated with it when we launched kigo and which is still out there and it's a tutor for students it's a teaching assistant for teachers but what we're launching today as part of this partnership is these state-of-the-art teacher tools we're going to be able to give free to every teacher in the United States so that they can get at productivity improvements yeah it big big deal big deal big I I I actually think teaching will be the the first mainstream profession to really benefit from uh generative AI lesson planning progress reports grading papers etc etc and I think if we can win teachers hearts and Minds then it gives us that much a better chance of also being able to reach students so one last thing before we go uh I know that you have uh just written a book uh and like having written a book myself like it's uh like a lot a lot of work so why and like everybody body should like you know pay attention to these uh fantastic endorsements the book has gotten and like go read a copy of it it's like a fascinating work not just about education and your mission but uh like I think also it has a bunch of really interesting lessons about how you can ambitiously use AI to solve heart problems so but but why did you choose to write a book now yeah you know this is the second book that I wrote the first book I wrote back in 2011 and I remember when uh the the publisher had reached out to me then I was like why would I write a book I could just put it all on YouTube and you know share it with the world but there's something about a writing writing a book that lets you frame the problem and I felt that 10 years ago when or 12 years ago when KH Academy first came on the scene and I think this moment we all feel even a little bit more overwhelmed I mean you mentioned these inflection points this exponential growth we're all feeling like things are changing every week and I wanted to take the time to for myself understand where we are and where we're going and hopefully a reasonably Timeless way that's not going to be dependent on whatever the frontier model of the day is and I think especially anyone who cares about education cares about work cares about what skills their kids should learn to keep up how we can all be more productive what the future of Admissions and of recruiting is going to look like hopefully this will be useful for them too yeah well I am incredibly grateful for the partnership that we have and more importantly for the work that you're doing in the world so thank you so much for being with us todayin thank [Applause] you so another really incredibly impactful area where these new AI tools and platforms are going to have an enormous impact is healthc care um so I just wanted to share another personal anecdote uh with you all uh so like I grew up in rural Central Virginia um and my mom and brother and most of my family still live in the place where I grew up um my mom is like a 74 year- old southern woman and uh has been suffering from a thyroid condition for 26 years which you know entirely under control like she's been taking medication uh for it um and just last fall she had uh like some sort of change in her uh like in her system uh where like all of a sudden her medication wasn't working as well as it used to which resulted in her spending a bunch of time in the hospital like trying to figure out what was going on and the Health Care system in the part of the world where my mom lives is like super overburdened um like it it's you know not a place where like tons of people are sort of moving for Economic Opportunity like it's one of those places where people tend to move away from to seek Economic Opportunity which has impacts on everything that is happening in that part of the world and I sort of uh as I was trying to help my mom navigate this situation sort of I was looking at how could AI have helped relieve the suffering that she was experiencing and you know if doctors everywhere had access to these tools like a lot of what she went through could have been immediately alleviated like if you just take her symptoms that she was presenting and put it into gp4 uh along with her chart it would immediately say like go get this test uh which is like going to help find root cause uh and then the results of the test like could be input into the same session uh which would then sort of give a set of recommendations to doctors about course of therapy and like if that had happened in her case like what was six visits to the hospital like could have been compressed to one um and you know I I think about this a lot because I I worry even whether or not my mom would have pulled out of a health spiral that she was in if she hadn't had me for a son intervening on her behalf and I wor about all of the people in the world who don't have someone to intervene on their behalf uh like who are interacting with one of these resources that is overburdened and so like I just want us all to think about as we're imagining what the set of possibilities are for you know what we go do with AI like things like what Saul is doing with Khan Academy uh and like some of the amazing potential that we have to reduce suffering in the world and to like help make uh you know things like high quality healthare more Equitable and accessible like just super inspiring to think about so you know the last thing that I want to chat about uh before we uh get to a conversation with Sam Alman is how we at Microsoft have been thinking about building applications on top of this incredible platform that uh is emerging right now and so the challenge I think for us and I think it's the same challenge that all of you face is that you really want to focus on things that have made the transition from impossible to mirly difficult like that's where all the interesting stuff is like if you look at the history of platform revolutions like that's where all the interesting companies uh emerge from it's where all the Innovation happens it's where all the value gets unlocked um and in the case of Technology platforms that are sort of exponentially progressing it's like the only reasonable place to go aim because if you're aiming somewhere different uh like the platform is becoming so much more capable and so much cheaper over time that everything that you sort of have in your imagination that's too expensive to do right now or too fragile is going to become cheap and robust before you can even blink your eye and so like that is really you know the the thing more than anything else that I would say to all of you to take away from what I'm saying here today is like really focus on those phase transitions so while you all have been out there you know grinding away building really extraordinary things over the past year with all of these AI tools that are coming like we've been hard at work trying to make forward progress on our AI platform so yeah we talked a lot about how we're optimizing the current Frontier like making things cheaper making them more powerful and complete uh but we've also been hard at work building new super Computing infrastructure and working with uh our partners at open AI to push that Frontier forward and like we showed this slide at the beginning like there's this like really beautiful relationship right now between this sort of exponential progression a compute that we're applying to building the platform to the capability and power of the platform that we get and I just wanted to you know sort of without without mentioning numbers uh which is sort of hard to do to give you all an idea the scaling of these systems so in 2020 we built our first AI supercomputer for open AI uh it's the supercomputing environment that trained gbd3 and so like we're going to just choose Marine Wildlife as our scale marker so you can think of that system uh about as big as a shark so the next system that we uh built um scale-wise is about as big as uh a Orca uh and like that is the system in uh that we delivered in 2022 that trained GPT 4 the system that we have just deployed is uh like scale-wise uh about as big as a whale relative to like you know this shark siiz supercomputer and this Orca siiz supercomputer and it turns out like you can build a whole hell of a lot of AI with a whale siiz supercomputer um and and so you know one of the things that I just want everybody to really really be thinking clearly about and like um this is going to be our segue to talking with Sam is the next sample is coming so like this whale siize supercomputer is hard at work right now building the next set of capabilities that we're going to put into your hands so that you all can do the next round of amazing things with it and so with that I'd like to bring Sam Alman to the stage hey good to see you you too so uh you are one of the busiest people on the planet wild week it's yeah it's a wild week it's a wild year man um but so I I really appreciate you taking uh time out to chat with us today um so I I guess what I really wanted to start our conversation uh about and like I asked you this question last week is you know there there's just been an extraordinary amount of change over the past year and a half year uh like what has been the thing that has surprised you most uh like particularly relevant into an audience of developers I mean I'm delighted to be here uh and obviously great to see you but developers have been such a core part of what's been happening this last year and a half um there's millions of people building on the platform what people are doing is totally amazing and the speed of adoption and talent and figuring out what to build with all of this over what has really not been very long like when we put gpt3 out in the API uh some people thought it was cool but it was where and seeing what people have done with gp4 and seeing now what's happening with GPT 40 even though it's new and hasn't been out that long uh is quite remarkable I've never seen a technology get adopted so quickly and such a meaningful way uh the what people are building how people are finding out how to do things that we never even thought of possible which is why it's always great to have an API uh that's been very cool to see yeah and I I think you know what you just said is like one of the the most important points to me like there there's a version of AI that could have existed that is uh you know like a bunch of smart people like building uh you know things at extraordinary scale and then just building it into a bunch of products where everybody gets to passively use them like the the really brilliant thing that you all have done is like taken the exact same set of things and like decided to make it available to like any developer who's able to sign up for an API key yeah we we try to be really thoughtful about what makes a good API for this there's going to be all kinds of ways people can use this but the more this can just be a layer that gets built into every product every Service uh the better and we've tried to make it such that if you want to add intelligence to whatever you doing uh any product any service we make that very easy yeah and like again I think the progress has been stunning so you know I I think you know the the setup for uh like introducing you on to the stage here was uh I saw that big wh yeah like you know you're you're making good use of the whale siiz computer right now and so like I without like getting too specific which we can't be obviously like what are the category of things that people should be expecting over the next you know K months so the the most important thing and this sounds like the most boring obvious Tri thing I can say but I think it's actually much deeper than it sounds the most important thing is that the models are just going to get smarter generally across the board there will be a lot of other things too which we can talk about but if you think about what happened from GPT 3 to 3.5 to 4 it just got smarter and you could use it for all these things it got a little more robust it got much safer uh both because the model got smarter and we put much more work into building the safety tools around it um it got more useful but the underlying capability this amazing emergent property of like we actually are are seeming to increase the general capability of the model across the board that's going to keep happening and the the jump that we have seen in the utility that a model can deliver with each of those half step jumps and smartness it's quite significant each time so as we think about the next model in the next one and the incredible things that developers are going to build with that I think that's the most important thing to keep in mind uh also speed and cost really matter to us so with GPT 40 we were able to bring the price down by half and double the the speed um new modalities really matter uh voice mode has been actually a genuine surprise for me in how much I like the new voice mode and I when people start integrating that I think that'll matter but but it's the overall intelligence that'll be coming that I think matters the most so you for a while now have been one of the most successful startup investors uh in the world um and like now you are one of the most successful uh CEOs of one of the most important companies in the world is so you got a room full of developers here you know like I think there are 5,000 people in the room and there about 200,000 people online right now um what's your advice to them is like they think about how to spend their precious time given what's happening in the world like what what's your advice two two things number one uh this is probably the most exciting time to be doing building a product doing a startup whatever it is uh that we have seen at least since the mobile boom um and probably I would say since the internet and maybe even bigger than that we don't know yet yeah um but the the big opportunities the big you know the ability to sort of build something new and really kind of like change the landscape that comes at the platform shift times and we haven't had a platform shift in a while and this looks like it's really truly a platform shift uh and so my biggest piece of advice is like this is a special time and take advantage of it this is like not the time to delay what you were planning to do or wait for the next thing like this is a special moment uh in a few years where a lot of stuff is going to happen and a lot of like great new things are going to get going um the second thing also about platform shifts is when the mobile phone Revolution started or really got going like 2008 2009 you would see people say um we're a mobile company you know we're have a mobile app and then only a few years later no one said they were a mobile company because it was like table stakes and an amazing new technology which I would bias but we put AI in that category uh it doesn't get you out of the hard work of building a great product or a great company or a great service um you still have to do it AI alone is a new enabler but it does not automatically break the rules of business and so you can use this as like a new thing to do but you still have to figure out how you're going to build uring value and whatever you're doing um and it's easy to lose sight of that in the excitement of the Gold Rush yeah so One Last Thing Before we let you go so you know you and I and like members of your team and members of the Microsoft team have been doing really an extraordinary volume of work over the past uh year and a half two years thinking about safe deployment of an awful lot of AI capability like everything from you know apis and developer tools to end products uh and you know I think we you know have accumulated a really interesting volume of experience like experience that sort of hard to get if you're not doing deployments at this scale um so I I you know and I think you just mentioned something that's like really really interesting like part of uh you know part of the interesting and surprising progression of capabilities of these models means that they're more useful in like helping to like make AI systems safer so I I don't know whether you had some thoughts you wanted to share there as well you know when we first developed this technology we spent a lot of time talking about all right we've made this thing it's cool are we ever going to be able to get it to an acceptable level of robustness and safety and now we kind of take that for granted with gp4 um you know if you use it it's far from perfect we have more work to do but it is generally considered robust enough and safe enough for a wide variety of uses and that took an enormous amount of work across both teams and fundamental research like when we started this we're like we've got this thing we've got this language model it looks like kind of kind of impressive and kind of not and even then how are we going to like get it aligned and um what it what does it me you know what is it going to take to be able to deploy it the number of different teams we've had to build up uh to go from research and creation of the model to Safety Systems to figuring out policy to how we do the monitoring um that's a huge amount of work but it's it's necessary uh to be able to deploy these and use them like you know when you take a medicine you want to know it's going to be safe when you use an AI model you want to know it's going to be robust behave the way you want and have been super proud of the work that teams have done together and I think it's amazing how fast this much work has happen and that we can all now use this and say oh yeah it basically it basically works as the models get more powerful there will be many new things we have to figure out as we move towards AGI um the level of complexity and I think the new research that it'll take will increase I'm sure we'll do that together but uh we view this as a gate on being able to put these things out into the world which we really want to do yeah it's definitely table sakes so thank you so much for uh for being with us here today like I really appreciate your time uh it's awesome to hear from you awesome all right so I will I I think this is all that's separating you all from the rest of your build and probably lunch uh so like my very last uh thing for you all is the following call to action so Microsoft and our partners like open AI are spending an extraordinary amount of energy and investing like Capital at an unprecedented scale trying to make sure that we are building a genuinely valuable platform but like all we're doing is building the platform and like I don't even think that that's the most important part of the AI Revolution that's happening right now it's you who are doing the work like you're the ones who are making all of these things matter um like we could build platforms all day all night and if you all didn't have the great ideas like you didn't understand the consequential impact that you wanted to have on the world it would all be for nothing and so I'm incredibly grateful for all of the things that you all have done on the platform over the past year and I am incredibly excited to see what you all are going to go do in the year ahead thank you all so [Applause] much e e