How to Build Human-in-the-Loop for AI Agents (Practical Guide)

Dave Ebbelaar (LLM Eng) · Intermediate ·🤖 AI Agents & Automation ·3mo ago
Want to start freelancing? Let me help: https://go.datalumina.com/7DnHzBg Want to learn real AI Engineering? Go here: https://go.datalumina.com/bnyLApI 💼 Need help with a project? Work with me: https://go.datalumina.com/TMGbUvO 🔗 GitHub Repository https://github.com/daveebbelaar/ai-cookbook/tree/main/models/openai/10-human-in-the-loop ⏱️ Timestamps 00:00 Human-in-the-Loop in AI 02:17 Python-Based Approaches 04:13 Tool Calling vs. Routing 07:50 Implementing Approval Steps 10:15 Production Patterns Overview 12:33 SSE Streaming for Real-Time Apps 19:14 Async Process for Backend Workflows 22:59 Key Concepts Recap 📌 Description Learn how to implement human-in-the-loop for your AI agents with two Python approaches: LLM routing with structured output and tool call interception. I cover two production patterns; SSE streaming for real-time chat apps and async workflows with notifications, showing how to manage state, deferred execution, and stateless resume. Perfect for AI engineers building reliable agentic systems that require human approval before executing sensitive actions. 👋🏻 About Me Hi! I'm Dave, AI Engineer and founder of Datalumina®. On this channel, I share practical tutorials that teach developers how to build production-ready AI systems that actually work in the real world. Beyond these tutorials, I also help people start successful freelancing careers. Check out the links above to learn more!
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Dave Ebbelaar · Dave Ebbelaar · 0 of 60

← Previous Next →
1 How to Install Homebrew on Mac (Getting Started)
How to Install Homebrew on Mac (Getting Started)
Dave Ebbelaar
2 How to Install Python on Mac (Homebrew)
How to Install Python on Mac (Homebrew)
Dave Ebbelaar
3 How to Install Anaconda on Mac (Getting Started)
How to Install Anaconda on Mac (Getting Started)
Dave Ebbelaar
4 How to Set up VS Code for Data Science & AI
How to Set up VS Code for Data Science & AI
Dave Ebbelaar
5 How to Use Git in VS Code for Data Science
How to Use Git in VS Code for Data Science
Dave Ebbelaar
6 Data Science Desk Setup to Maximize Productivity
Data Science Desk Setup to Maximize Productivity
Dave Ebbelaar
7 THIS Is How I Write Clean Data Science Code EVERY TIME
THIS Is How I Write Clean Data Science Code EVERY TIME
Dave Ebbelaar
8 Data Science Tutorial - Project Structure
Data Science Tutorial - Project Structure
Dave Ebbelaar
9 Changing rcParams for Better Data Science Plots | Matplotlib Tutorial
Changing rcParams for Better Data Science Plots | Matplotlib Tutorial
Dave Ebbelaar
10 How to Read Excel Files with Python (Pandas Tutorial)
How to Read Excel Files with Python (Pandas Tutorial)
Dave Ebbelaar
11 My Data Science Journey (Zero to Freelance)
My Data Science Journey (Zero to Freelance)
Dave Ebbelaar
12 How I Automate Data Visualization in Python
How I Automate Data Visualization in Python
Dave Ebbelaar
13 16 Apps I Use Daily as a Data Scientist
16 Apps I Use Daily as a Data Scientist
Dave Ebbelaar
14 How to Manage Conda Environments for Data Science
How to Manage Conda Environments for Data Science
Dave Ebbelaar
15 How to Export Machine Learning Models in Python
How to Export Machine Learning Models in Python
Dave Ebbelaar
16 VS Code Speed Hack for Data Science
VS Code Speed Hack for Data Science
Dave Ebbelaar
17 17 VS Code Tips That Will Change Your Data Science Workflow
17 VS Code Tips That Will Change Your Data Science Workflow
Dave Ebbelaar
18 How to Predict the Future with Python (Forecasting Tutorial)
How to Predict the Future with Python (Forecasting Tutorial)
Dave Ebbelaar
19 How to Use Python Environment Variables
How to Use Python Environment Variables
Dave Ebbelaar
20 7 Data Science Tips for Beginners in 2023
7 Data Science Tips for Beginners in 2023
Dave Ebbelaar
21 How to Effectively Use the Data Science Lifecycle
How to Effectively Use the Data Science Lifecycle
Dave Ebbelaar
22 Full Machine Learning Project — Coding a Fitness Tracker with Python (Part 1)
Full Machine Learning Project — Coding a Fitness Tracker with Python (Part 1)
Dave Ebbelaar
23 Full Machine Learning Project — Processing Raw Data (Part 2)
Full Machine Learning Project — Processing Raw Data (Part 2)
Dave Ebbelaar
24 Full Machine Learning Project — Data Visualization with Matplotlib (Part 3)
Full Machine Learning Project — Data Visualization with Matplotlib (Part 3)
Dave Ebbelaar
25 This Will Change Data Science as We Know It (ChatGPT)
This Will Change Data Science as We Know It (ChatGPT)
Dave Ebbelaar
26 Full Machine Learning Project — Detecting Outliers in Sensor Data (Part 4)
Full Machine Learning Project — Detecting Outliers in Sensor Data (Part 4)
Dave Ebbelaar
27 Full Machine Learning Project — Low-pass Filter & Principal Component Analysis (Part 5a)
Full Machine Learning Project — Low-pass Filter & Principal Component Analysis (Part 5a)
Dave Ebbelaar
28 Full Machine Learning Project — Fourier Transformation & Clustering (Part 5b)
Full Machine Learning Project — Fourier Transformation & Clustering (Part 5b)
Dave Ebbelaar
29 Full Machine Learning Project — Predictive Modelling (Part 6)
Full Machine Learning Project — Predictive Modelling (Part 6)
Dave Ebbelaar
30 Automate Machine Learning with ChatGPT
Automate Machine Learning with ChatGPT
Dave Ebbelaar
31 Scraping Web Datasets for Data Science Projects
Scraping Web Datasets for Data Science Projects
Dave Ebbelaar
32 Full Machine Learning Project — Counting Repetitions (Part 7)
Full Machine Learning Project — Counting Repetitions (Part 7)
Dave Ebbelaar
33 How to Use GitHub Copilot for Data Science (Python + VS Code)
How to Use GitHub Copilot for Data Science (Python + VS Code)
Dave Ebbelaar
34 Every Beginner Data Scientist Should Understand This
Every Beginner Data Scientist Should Understand This
Dave Ebbelaar
35 Revealing My New AI-Powered Data Science Workflow
Revealing My New AI-Powered Data Science Workflow
Dave Ebbelaar
36 Auto-GPT Tutorial - Create Your Personal AI Assistant 🦾
Auto-GPT Tutorial - Create Your Personal AI Assistant 🦾
Dave Ebbelaar
37 Build Your Own Auto-GPT Apps with LangChain (Python Tutorial)
Build Your Own Auto-GPT Apps with LangChain (Python Tutorial)
Dave Ebbelaar
38 Building Slack AI Assistants with Python & LangChain
Building Slack AI Assistants with Python & LangChain
Dave Ebbelaar
39 ChatGPT Code Interpreter - Goodbye Data Analysts?
ChatGPT Code Interpreter - Goodbye Data Analysts?
Dave Ebbelaar
40 How to Deploy AI Apps to the Cloud with Flask & Azure
How to Deploy AI Apps to the Cloud with Flask & Azure
Dave Ebbelaar
41 How to Build an AI Document Chatbot in 10 Minutes
How to Build an AI Document Chatbot in 10 Minutes
Dave Ebbelaar
42 Is Falcon LLM the OpenAI Alternative? An Experimental Setup with LangChain
Is Falcon LLM the OpenAI Alternative? An Experimental Setup with LangChain
Dave Ebbelaar
43 GPT Engineer... Generate an entire codebase with one prompt
GPT Engineer... Generate an entire codebase with one prompt
Dave Ebbelaar
44 Pandas DataFrame Agent... the future of data analysis?
Pandas DataFrame Agent... the future of data analysis?
Dave Ebbelaar
45 OpenAI Function Calling - Full Beginner Tutorial
OpenAI Function Calling - Full Beginner Tutorial
Dave Ebbelaar
46 How to use ChatGPT's new “Code Interpreter” feature
How to use ChatGPT's new “Code Interpreter” feature
Dave Ebbelaar
47 LangChain just launched their new "LangSmith" platform
LangChain just launched their new "LangSmith" platform
Dave Ebbelaar
48 How I'd Learn AI (if I could start over)
How I'd Learn AI (if I could start over)
Dave Ebbelaar
49 I Used AI To Scrape The Web & Write PDF Reports
I Used AI To Scrape The Web & Write PDF Reports
Dave Ebbelaar
50 LangSmith Tutorial - LLM Evaluation for Beginners
LangSmith Tutorial - LLM Evaluation for Beginners
Dave Ebbelaar
51 7 Lessons for New AI Engineers - Beginner’s Guide
7 Lessons for New AI Engineers - Beginner’s Guide
Dave Ebbelaar
52 The Rise of the "New-Age" Machine Learning Engineer
The Rise of the "New-Age" Machine Learning Engineer
Dave Ebbelaar
53 OpenAI Assistants Tutorial for Beginners
OpenAI Assistants Tutorial for Beginners
Dave Ebbelaar
54 How To Connect OpenAI To WhatsApp (Python Tutorial)
How To Connect OpenAI To WhatsApp (Python Tutorial)
Dave Ebbelaar
55 How to Build Chatbot Interfaces with Python
How to Build Chatbot Interfaces with Python
Dave Ebbelaar
56 PostgreSQL as VectorDB - Beginner Tutorial
PostgreSQL as VectorDB - Beginner Tutorial
Dave Ebbelaar
57 My MacBook Setup (as a coder & business owner)
My MacBook Setup (as a coder & business owner)
Dave Ebbelaar
58 Easiest Way to Connect AI Chatbots to WhatsApp
Easiest Way to Connect AI Chatbots to WhatsApp
Dave Ebbelaar
59 ClickUp Tutorial - What Is ClickUp Brain? 🧠
ClickUp Tutorial - What Is ClickUp Brain? 🧠
Dave Ebbelaar
60 My Development Workflow for Data & AI Projects
My Development Workflow for Data & AI Projects
Dave Ebbelaar

Related AI Lessons

The First Batch Payment Skill for Vercel Agents (skills.sh)
Enable batch crypto payments across 15 blockchains with a single command using the Spraay Protocol agent skill on Vercel
Dev.to AI
I kept seeing people ask if OpenClaw is secure, but the real email risk is way more boring
Understand the real email risk with OpenClaw is not security, but permissions and workflow configuration
Dev.to AI
10 free AI agent tools with no API key required
Discover 10 free AI agent tools that don't require an API key, and learn how to use them with a simple curl command
Dev.to AI
Your AI Agent Just Ran `rm -rf /` in Production — Here's How to Prevent It
Learn how to prevent AI agents from causing catastrophic errors in production, such as running `rm -rf /`, by implementing proper governance and security measures
Dev.to AI

Chapters (8)

Human-in-the-Loop in AI
2:17 Python-Based Approaches
4:13 Tool Calling vs. Routing
7:50 Implementing Approval Steps
10:15 Production Patterns Overview
12:33 SSE Streaming for Real-Time Apps
19:14 Async Process for Backend Workflows
22:59 Key Concepts Recap
Up next
Why Block gave Goose to the Agentic AI Foundation
The New Stack
Watch →