A voice that sounds good for one conversation is the easy part. Staying reliable across thousands of real, messy ones is the actual work.
The bare minimum
Full room, demo built in 30 minutes, no edits, no second chance. The agent takes the call and books a job. Every voice AI can do this. It's where the story starts, not where it ends.
Host: Hey, you mentioned part way through that you've got a personal vendetta against live demos. Do you wanna do a live demo?
Owen: Let's do it. Alright, hopefully there's no interference with the audio here, and I can give you a common path to what a demo sounds like.
Agent: Hello, this is Chloe from Demo Plumbing, on a recorded line. How can I help you today?
Owen: Hey Chloe, I've got a leaking tap. I wanna book someone out to come and fix it.
Agent: Got it, we deal with leaking taps all the time. I'll grab a few details so we can get someone out to you. What's your full name?
Owen: Owen White.
Agent: Thanks Owen. What's the address where the tap is leaking?
Owen: 123 Test Road, Majestic.
Agent: Got it. What time would you like the plumber to come out?
Owen: Uh, as soon as possible.
Agent: We can get someone out as soon as possible. Is there a specific time that works best for you, or would you like us to send someone in the morning?
Owen: Uh, I guess someone in the morning.
Agent: Sure, I can book you in for the morning. What time works best for you?
Owen: As soon as possible.
Agent: I understand you'd like someone as soon as possible. Let me check our availability. We have a plumber available tomorrow morning at 9:00am. Would that work for you?
Owen: Yeah, that works.
Agent: Great, I've got you booked in for tomorrow at 9:00am. Is the number you're calling from the best contact number?
Owen: Yes it is.
Agent: Perfect, Owen. I've got you all set for tomorrow at 9:00am at 123 Test Road, Majestic. Our plumber will call you on arrival. Is there anything else I can help you with?
Owen: No that's all, thank you very much.
Agent: Great, thanks for your time. Have a great day.
The interruption. The special request. The background noise. The "actually, make that two." Every call has a moment no script saw coming, and this is how we handle it.
Why Your 'Extra Tomato' Order Breaks Most AI
Common requests are easy. Customer trust is won in the exceptions.
So the other thing you're gonna see with AI is that the common request is easy, but trust is won in the exceptions.
So, another fast food example. Not sure if anyone's been to the Burger King drive thru at Curletts Road, but they have an AI that takes your order. So if you go there and you know what you want, Whopper, fries, Coke, it'll handle your request, no problem.
But the minute you start hesitating and you say, oh actually, let me get extra tomato. Um, can you make it a large. Uh, this date's not going very well, so let's split the payment. I've also got a voucher as well, so I'll use it, I'll give you the code.
Unless you've architected these edge cases, the likelihood of that system breaking is very high.
Voice AI's 2 Biggest Bottlenecks
The two places it actually falls over, and why they have nothing to do with the voice.
Host: That's cool though. I mean, what are some of the things that can go wrong in a case like that? Cause that's really handy if you're like a plumbing company, right? Then you can just have this Chloe on call all the time.
Owen: Yeah. So I guess there's two things at the moment with voice AI that is the big bottlenecks, that I don't think anyone's been able to answer.
One is balancing latency with intelligence. So the faster you want it, the less smart it's gonna get. But the smarter you want it, the less, you know, response time you're gonna get.
And then the other one is, you know, audio quality. So again, if there's interference, between, you know, people talking, there's like technology, it might misinterpret what you say, and then that affects the speech to text, and then, you know, it causes these loops. So those are the two major, I guess, bottlenecks with voice AI.
And I think the models are gonna get better and better, but as of now it's just about live data and seeing how you can streamline the process as much as you can. Like for example, instead of asking me what's your number, and I have to say 021 whatever whatever, and it has to repeat it, it just says, is the number you're calling from the best number? Cause 90% of the time someone calling, that is their best number. And then on the back end in the system, it'll take that deterministic number and use that as the data. So it makes it way more reliable and less likely to break.
The Hardest Part of Voice AI
It was never the talking. It is everything the agent has to check before it answers.
The unscripted edge cases is where you're gonna spend like 80 to 90% of your time.
Cause when you build it, you have the common path in mind, right? You know what outcome you want it to achieve. You build it, you test it, great, it works. Then you try to harden it with edge cases, cause you know the weaknesses of the agent. You build it, cool.
As soon as you give it to someone else, the way they speak, the speed, their accent, the order they say things in, it just opens up a Pandora box of different edge cases.
So for that reason, having live data is really important. But you also need a system that works in the background, that analyses every single call, and then it iterates and improves after each call.
The Reality of Voice AI Demos
You don't get to pick the conditions of a real call. Calls from a job site, a moving car, a noisy kitchen, half of them on speaker. The agent has to understand them anyway.
That is exactly why I have a personal vendetta against YouTube demos. When we first started demos, mate, it's like you always have the clearest audio, one clean question, there's no consequences.
But as soon as you start building it, you have accents, you have interruptions, you have the difference between a phone call and a web app. The frequencies they use is completely different, so the audio quality drops on a phone call. It messes with the speech to text.
And then you also have tool failures, if the app that you're using isn't in sync because it's just updated.
Trusting it to handle your line requires clarity on the rules behind it: it quotes what you've set, books only what you can actually deliver, and never tells a customer something you didn't approve.
The 3 Layers of AI Agents
How it talks, how it decides, where the work actually runs. The map behind a real agent.
The three layers of AI agents. We already talked about the front door, the interface, you know, the voice. Obviously you can have it for chat and SMS.
Um, the decision layer. Your large language model, aka LLM, is like your GPT 5.5, your Mythos, your DeepSeek, Grok. These are all just the brain of the decision layer. These models are all trained on different knowledge, instructions and rulesets. If you want it to be trained on your business, then you would need to use these models, to then give it the data, to then essentially create your own brain.
The action layer is when you're sitting at your desk, you're typing, you're clicking whatever apps you guys use. All of that can be automated.
So when you combine the action layer and the decision layer, as well as an interface, then that's when people actually start, you know, being scared about losing their jobs.
You: Architect. AI: Technician.
With a new AI tool released every other day, the highest-return skill is explaining your own SOPs plainly enough for a 5-year-old to follow.
So with that understanding and the difference in mind, the best thing you can do tomorrow when you're sitting at your desk is picture you have an apprentice that never sleeps, never forgets.
Think about the one single workflow or task that you want to automate. How are you gonna explain to that apprentice, step by step, what you want it to do in order to achieve that task?
And then the clearer that you can conceptualize that, the closer you're gonna get to the reality of actually being able to fully automate that task.
One day Jess, she's working for this startup, and she needed to call a plumber. Um, so she spent the entire day dialing, and then the calls would either all go to voicemail, or they'd say the same thing, which is basically, well, we need to come out and see it before we give you a quote.
And she told me this, and I was like, man, this is, it's 2025, like, we have AI, we have all this technology. I personally hate being on hold. I hate voicemail. There has to be a more efficient way.
And at the time, I don't think ChatGPT even had voice yet, but we knew it was possible. So we started researching, we started building. Um, and that's how we got started with voice AI.
Tired of the phone letting jobs walk? Let's talk about what a reliable agent looks like for you.