Great voice agents can save companies millions of dollars. Badly built ones can cost just as much.
Say you have access to the best model in the world. It talks exactly like a human, switches languages, even sings. Latency is low and it understands speech perfectly. All of that goes in the bin if the harness around it is weak.
One wrong digit is a breach
Take an application in the EU, or a HIPAA use case in the US, where you have to handle personal data carefully. A caller gives your agent their phone number. One time in 100, the agent gets it wrong. In reality the rate is worse, but let’s say 1 in 100.
Your system then sends an SMS with private details to that number. That’s a breach, and you pay a lot of money. (More on that side of things in HIPAA and GDPR compliance for voice AI.)
That’s why new model launches don’t excite me much anymore.
Two years ago I built a voice agent on Claude Sonnet 3.7. The latency was freaking awful. The STT accuracy was awful too, and the tool calling was rough. I asked a lot of people to test it, and they said they couldn’t jailbreak it and the conversation was surprisingly decent. None of that came from the model. The harness around it was good.
It’ll keep happening. Every launch implies that the new model solves everything. It doesn’t. You have to do things the right way and put more care into your agents.
What a good agent looks like
A good agent:
- understands intent. From what the caller said, it knows what they want, whether to call a tool, and what to do next.
- is functionally correct. It can complete a task end to end.
- follows instructions and doesn’t go off the rails.
- handles context and state well, even on long conversations.
- is pleasant to talk to. It doesn’t ramble and says what it means precisely.
- is safe for your system.
The last one matters most. The agent should be very hard to jailbreak. Making it impossible almost never works, so it should also be built so that it can’t do real harm to your system even after someone jailbreaks it.
Step 1: Call your own agent
Dogfooding is a corporate term and I don’t love it, but it’s accurate. Call your agent very often, and make sure everything works from end to end. Test it yourself, not only with AI.
AI testers don’t pick up some very obvious bugs. Calling it yourself catches them and saves you a lot of time.
Step 2: Automated tests in three buckets
I split automated tests into three kinds.
Deterministic tests. If I call this tool with these parameters, X happens. These must always pass.
Slightly non-deterministic tests. I prepare a mock conversation where it’s obvious what the agent should do, and check that it does it. They catch simple regressions and confirm everything still works as intended.
Fully non-deterministic tests. I simulate conversations with AI, throw in random words and garbage transcriptions, and see whether the agent still performs. I don’t expect all of these to pass, only most of them. The failures show me where the agent is weak and where to put more effort, and the overall pass rate tells me how we’re doing on average.
Step 3: Guardrails and the harness itself
The last step is sanity-checking everything and building the actual harness. I do this with Jev, which answers prepared yes/no questions very quickly and gives a probability. I can ask it, “Does this look like a malicious attempt to break the agent?” with the caller’s input, and get back a yes with a probability.
That works anywhere in the conversation. I use it to switch assistants, to pick the most relevant tools, and to build observability pipelines, sometimes with Jev alone, sometimes with an LLM, sometimes with both.
For every capability, ask yourself “what could go wrong if…?” Then build a guardrail around it, or design that part of the system differently.
Here’s a simple example. Your agent can book meetings on its own. One person calls 500 times: “I want to book this day. And this day. And this day.” There’s no cap on bookings per person, or per phone number. One caller can now wreck your entire system.
You don’t think about cases like that unless you put real care into your voice agents.
Want someone to stress-test your agent?
I’ve built voice agents full-time for the past two years, on systems handling 50 million calls a year. If your agent has done something on a live call you didn’t expect, book a free consultation and bring the call. I build production voice agents with this kind of harness. A tutorial on how to do it is in the newsletter.