How Microsoft Engineers Build AI: Building and Evaluating Agents
๐ Welcome to Episode 2 of How Microsoft Engineers Build AI!
In this episode, go behind the scenes with Microsoft engineers as they dive into how agents are built, evaluated, and deployed โ with a practical demo straight from engineering.
๐ก Learn how Microsoft thinks about agent architecture, real-world challenges in permissioning and control, shaping agent behavior, handling uncertainty, and evaluation best practices. Whether you're an AI developer, data scientist, or tech enthusiast, this is your front-row seat to how cutting-edge AI systems are engineered at scale.
๐ Chapters
00:00 โ Intโฆ
Watch on YouTube โ
(saves to browser)
Chapters (11)
Introduction
0:37
Speaker introduction
1:01
What did you create? What problem did you solve?
2:04
Demo
4:11
Why do we need agents?
6:57
Where does it fit into the picture? What challenges tend to come up?
13:31
What about permissioning and control? How do you balance flexibility with secu
16:20
How do you shape Agent behavior?
21:44
Uncertainty Threshhold Check: Epistemic uncertainty, aleatoric uncertainty, hu
25:15
Post-Deployment Feedback Loop
26:54
What are some evaluation best practices?
Playlist
Uploads from Microsoft Developer ยท Microsoft Developer ยท 0 of 60
โ Previous
Next โ
1
![What you need to know about LLMs [Pt 1]](https://i.ytimg.com/vi/GQ_2OjNZ9aA/mqdefault.jpg)
2
![Get started with AI for free using Phi-3 [Pt 6]](https://i.ytimg.com/vi/Ds32MS9SHzU/mqdefault.jpg)
3
![Azure tools & services for hosting and storing AI apps [Pt 9]](https://i.ytimg.com/vi/WB6Fpzhwyug/mqdefault.jpg)
4
![Run AI models on your local machine with Ollama [Pt 5]](https://i.ytimg.com/vi/dLfNnoPv4AQ/mqdefault.jpg)
5
![Streaming Generative AI output with the AI Chat Protocol [Pt 10]](https://i.ytimg.com/vi/fzDCW-6hMtU/mqdefault.jpg)
6
![Essential prompt engineering techniques [Pt 2]](https://i.ytimg.com/vi/gQ6TlyxBmWs/mqdefault.jpg)
7
8
![Improve AI accuracy and reliability with RAG [Pt 3]](https://i.ytimg.com/vi/xkFOmx5yxIA/mqdefault.jpg)
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
What you need to know about LLMs [Pt 1]
Microsoft Developer
Get started with AI for free using Phi-3 [Pt 6]
Microsoft Developer
Azure tools & services for hosting and storing AI apps [Pt 9]
Microsoft Developer
Run AI models on your local machine with Ollama [Pt 5]
Microsoft Developer
Streaming Generative AI output with the AI Chat Protocol [Pt 10]
Microsoft Developer
Essential prompt engineering techniques [Pt 2]
Microsoft Developer
Generative AI with JavaScript - Introduction
Microsoft Developer
Improve AI accuracy and reliability with RAG [Pt 3]
Microsoft Developer
Data As a Corporate Assetโthe GenAI-era Take (Part 2)
Microsoft Developer
Boost your local development with Dev Container templates for Azure SQL
Microsoft Developer
Boost your local development with Dev Container templates for Azure SQL
Microsoft Developer
What's your favorite use case for Copilot?
Microsoft Developer
What's your favorite use case for Copilot?
Microsoft Developer
How can Copilot help me make my software more accessible?
Microsoft Developer
How can Copilot help me make my software more accessible?
Microsoft Developer
Database performance and scalability with Azure SQL Database Hyperscale elastic pools | Data Exposed
Microsoft Developer
What languages are supported by Copilot?
Microsoft Developer
What languages are supported by Copilot?
Microsoft Developer
Azure Essentials: Improve the reliability, security, and performance of cloud and AI investments
Microsoft Developer
Use Copilots To Put Esoteric Tools to Work
Microsoft Developer
Data Engineering in Fabric using Copilot | Data Exposed: MVP Edition
Microsoft Developer
Mr. Maeda's Cozy AI Kitchen - AI Coaching the Coach, with Pat Cavanaugh
Microsoft Developer
GitHub Copilot for Azure: Discovering Azure services and AI templates
Microsoft Developer
GitHub Copilot for Azure: Deploying to Azure Kubernetes Service (AKS)
Microsoft Developer
GitHub Copilot for Azure: Exploring your Azure resources
Microsoft Developer
GitHub Copilot for Azure: Deploying to Azure with context
Microsoft Developer
GitHub Copilot for Azure: Understanding and managing your models in Azure AI
Microsoft Developer
GitHub Copilot for Azure: Planning a migration to Azure
Microsoft Developer
โHow does Copilot for Azure SQL work?โ and more of your questions answered! | Data Exposed
Microsoft Developer
How did you get started writing books for Windows and .NET?
Microsoft Developer
Streamline agreement workflows with Docusign
Microsoft Developer
Deploy OpenAI Services at Scale Using Provision Throughput Units
Microsoft Developer
Automated Deployment from AKS VsCode Extension
Microsoft Developer
Whatโs New in SQL Server & Azure SQL | Data Exposed Live @ PASS Summit 2024
Microsoft Developer
Microsoft 365 Copilot: Developer Camp
Microsoft Developer
Introducing Custom Engine Agents
Microsoft Developer
Unlock the Power of Custom Engine Agents
Microsoft Developer
Introducing Declarative Agents
Microsoft Developer
The Future of Declarative Agents
Microsoft Developer
Extending Declarative Agents
Microsoft Developer
Developer's Guide to Building Copilot Agents
Microsoft Developer
A Conversation with the Leaders Behind the Developer Tools
Microsoft Developer
Azure SQL Revealed 2nd Edition Book | Data Exposed
Microsoft Developer
Whatโs the origins of .NET generics?
Microsoft Developer
Essential Azure Skilling and Guidance
Microsoft Developer
Unlocking Multilingual Accessibility with Co-op Translator: A Case Study on Phi-3 Cookbook
Microsoft Developer
Master Azure Resiliency, Performance, and Security
Microsoft Developer
Mr. Maeda's Cozy AI Kitchen - Composite AI, with Hiromichi Kobashi
Microsoft Developer
Build Intelligent Apps on Azure
Microsoft Developer
Tell us a story from the era when you got your 5 (and 3!?!) year service award
Microsoft Developer
Next Gen Data Analytics with Microsoft Fabric
Microsoft Developer
Tell us a story from the era when you got your 10-year service award
Microsoft Developer
Building the ultimate chatbot on your own data with Azure SQL and Semantic | Data Exposed
Microsoft Developer
Trustworthy AI: From Principles to Practice
Microsoft Developer
Tell us a story from the era when you got your 15-year service award
Microsoft Developer
Tell us a story from the era when you got your 20-year service award
Microsoft Developer
Tell us a story from the era when you got your 25-year service award
Microsoft Developer
Tell us a story from the era when you got your 30-year service award
Microsoft Developer
Tell us a story from the era when you got your 35-year service award
Microsoft Developer
Tell us a story from the era when you got your 40-year service award
Microsoft Developer
DeepCamp AI