Brooke Hopkins, founder and CEO of Coval and formerly the lead of evaluation job infrastructure at Waymo, ends up arguing that agents will become reliable the way cloud infrastructure did: through simulation at scale, layered metrics, redundancy and monitoring, not through any single model becoming flawless. Her company sells that evaluation layer, starting with voice and chat agents, and she frames the human-in-the-loop as staying for now, with fully self-improving agents still premature.
From Waymo to agents. The host opens on Waymo, which he and his annual-predictions co-host named their "wow moment" of the year in episode 849. Hopkins says Waymo took about ten years to get from Mountain View to rider-only service in San Francisco two years before recording, and is now spreading faster to Los Angeles, Phoenix and other cities partly because simulation became accurate enough to replace driving logs. Coval applies the same idea to autonomous agents, anything that must respond to what the world does back: voice, chat, and eventually web-browsing agents.
Why manual testing fails. A multi-step agent is slow to test by hand. A phone call can take 15 minutes or more, and testers lose track of which pathways they have covered. Coval runs representative sets of pathways and counts how often agents get stuck, mispronounce, mistranscribe or make logic errors. Customers vary simulation determinism with temperature, rerun a failing prompt 10 or 100 times, and re-simulate from production transcripts, a workflow borrowed from replaying road logs. As at Waymo, simulation filters what humans should look at; it does not replace review.
The developer lifecycle and platform choice. Hopkins walks through a customer's path: build an MVP on a voice platform, iterate on the prompt without voice, add RAG or multiple agents, run 100 simulated conversations at once, review failures, turn what you look for by hand into metrics, then automate and monitor in production. She declines to name preferred voice-agent providers. Coval is meant to be framework-agnostic, and she sets out the trade-offs: low-code versus configurable, and instruction following, function calling, workflow following, naturalness and creativity.
Precision against scalability. The host frames a spectrum running from tens of thousands of hand-built conversation flows to unreviewed auto-generated tests. Hopkins recasts it as cost and latency against signal. Coval helps decide what runs on each PR, every six hours or nightly, and before releases, and shows whether test sets match production.
Cascading errors. Asked about a triage agent that calls the wrong tool early, she says multi-step evaluation differs from single-call evaluation because non-determinism compounds across pathways. Fixes include self-healing setups where a slower "overthinker" model watches a fast one, redundancy borrowed from aerospace and self-driving, and handoff to a human, which she says most Coval customers already have. Servers are unreliable, yet cloud systems reach six nines; she expects agents to follow. The host goes further, calling agents handling complex tasks "inevitable".
Metrics. Hopkins argues there is no objective line between good and bad conversations, so Coval layers metrics and looks at trends. Getting the most common case wrong outweighs 99% elsewhere. Examples include adherence to a JSON-described workflow (missing a phone number means the task is done but done wrong), reference-free checks of function-call arguments, unanswered-question detection, and agent-versus-human task time as a sign of going in circles. She thinks automated prompt and agent optimization is still too early.
Autonomy and the future. Hopkins separates autonomy in development from autonomy in execution. Rigid workflows break when a caller volunteers an email first, and she hopes agents move toward true autonomy, even reading API documentation themselves instead of being given functions.
Why voice first. Voice is more constrained, builds on existing call centres, and was exploding faster than any other agent type about six months before recording. She calls it a universal natural-language API between businesses and the first platform since mobile every company will be expected to have. Hopkins, a solo founder, credits Y Combinator with narrowing the company to voice around its first customer.