Build a Voice Agent in an Hour with Claude Code | AssemblyAI Workshop

AssemblyAI · Intermediate ·🧠 Large Language Models ·1mo ago

Key Takeaways

Builds a voice agent using AssemblyAI Voice Agent API and Claude Code

Original Description

In this live workshop, AssemblyAI Product Manager Dan Ince builds a fully working appointment booking voice agent from scratch, using only the AssemblyAI Voice Agent API and Claude Code. No pre-written code, custom tools or MCP servers. Just a docs URL and a few prompts. By the end, you'll have a working voice agent with a frontend UI, tool calling, and a deployment you can share with anyone. What you'll walk away with: -A working voice agent app you can run locally or deploy to Railway -The exact Claude Code prompts used to scaffold backend, frontend, and tool calling -A reusable architecture pattern (Python backend for token auth + HTML frontend) that keeps your API key off the client -Techniques for appointment booking, tool chaining, and progressive tool reveal -Practical tips on turn detection, echo cancellation, and reducing latency -Answers from the most popular building FAQs What you'll need: 🔑 A free AssemblyAI API key: https://www.assemblyai.com/products/voice-agent-api?utm_source=youtube&utm_medium=referral&utm_campaign=workshop&utm_content=claude_voice_agent 🤖 Claude Code (VS Code plugin or the Claude app) 🖥️ Docs link to drop in Claude Code: https://www.assemblyai.com/docs/voice-agents/voice-agent-api ⏱ Jump to the build: 11:39 — Live build starts: blank project in VS Code 13:35 — Prompt 1: Scaffold Python backend + HTML frontend with token auth 18:31 — Running the app locally for the first time 24:51 — First working voice agent in the browser 29:02 — Prompt 2: Add appointment booking tool call 35:40 — Prompt 3: UI redesign (car mechanic booking agent) 43:13 — Final live demo: voice agent handles a full MOT booking 45:20 — Progressive tool reveal: how to eliminate tool-calling hallucinations 47:18 — Turn detection tuning + echo cancellation tips 54:00 — Deploying to Railway and generating a shareable URL The full code is on GitHub: https://github.com/dan-ince-aai/apex-auto-voice-agent ▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬▬ 🖥️ Website: https:/
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Related Reads

📰
Building Production-Grade LLM Evaluation Pipelines: From Vibes to Metrics
Learn to build production-grade LLM evaluation pipelines to catch hallucinations before deployment and improve model reliability
Dev.to AI
📰
Why Every AI Engineer Should Learn Hugging Face
Learn how Hugging Face simplifies AI development and why it's a crucial tool for AI engineers to master
Medium · Machine Learning
📰
A bug in Qwen3-TTS taught me voice is biometric
A developer's experience with a bug in a voice cloning model highlights the biometric nature of voice, emphasizing security and privacy concerns
Dev.to · Daniel Nwaneri
📰
What is LoRA and how it lets anyone fine-tune a massive AI model on a single GPU
Learn about LoRA, a technique that enables fine-tuning of massive AI models on a single GPU, making it accessible to individuals and small teams
Medium · LLM

Chapters (10)

11:39 Live build starts: blank project in VS Code
13:35 Prompt 1: Scaffold Python backend + HTML frontend with token auth
18:31 Running the app locally for the first time
24:51 First working voice agent in the browser
29:02 Prompt 2: Add appointment booking tool call
35:40 Prompt 3: UI redesign (car mechanic booking agent)
43:13 Final live demo: voice agent handles a full MOT booking
45:20 Progressive tool reveal: how to eliminate tool-calling hallucinations
47:18 Turn detection tuning + echo cancellation tips
54:00 Deploying to Railway and generating a shareable URL
Up next
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)
Watch →