Adventures in AI: Deploying and inferencing open source and custom models on K8s​ | BRK194

In the rapidly evolving field of AI, deploying LLMs efficiently and at scale is a significant challenge. Explore the process of containerizing and deploying open-source and custom LLMs on AKS. This session will guide developers through starting from a managed namespace, incorporating tools like KAITO, and utilizing the VS Code extension for KAITO and GitHub Copilot for Azure. To learn more, please check out these resources: * https://aka.ms/build25/plan/AzureAIEngineerCertification * https://aka.ms/aks/new * https://learn.microsoft.com/en-us/azure/aks/ai-toolchain-operator * https://developer.microsoft.com/en-us/reactor/series/S-1528/ 𝗦𝗽𝗲𝗮𝗸𝗲𝗿𝘀: * Sachi Desai * Jorge Palma 𝗦𝗲𝘀𝘀𝗶𝗼𝗻 𝗜𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻: This is one of many sessions from the Microsoft Build 2025 event. View even more sessions on-demand and learn about Microsoft Build at https://build.microsoft.com BRK194 | English (US) | Cloud Platform Related Sessions: BRK193 -- https://build.microsoft.com/sessions/BRK193?wt.mc_id=yt_PLlrxD0HtieHgukvOrEw3CqZuKtxiu_wnM LAB342 -- https://build.microsoft.com/sessions/LAB342?wt.mc_id=yt_PLlrxD0HtieHgukvOrEw3CqZuKtxiu_wnM LAB345 -- https://build.microsoft.com/sessions/LAB345?wt.mc_id=yt_PLlrxD0HtieHgukvOrEw3CqZuKtxiu_wnM #MSBuild, #CloudPlatform Chapters: 0:00 - Introduction of speakers and note on absent Co-speaker 00:00:36 - Previous AI and Kubernetes discussions 00:16:13 - Introduction to Kubernetes AI Tool Chain Operator 00:16:25 - Details on Kaito's Engineering Team Contribution 00:17:06 - Workspace Controller's Role in Resource Reconciliation 00:30:00 - Introduction to Quen 25 Coder model for code reviews 00:30:42 - Configuring GitHub application for PR review automation 00:45:48 - Deployment Flexibility with Kubernetes Manifests 00:54:56 - Utilization of Microsoft Co piloter for Azure