Flux → GPT-4.1 → ElevenLabs Flash is ahead on reliability, success rate, agent responses, data accuracy, stalled calls and response time, and Amazon Nova 2 Sonic on interruption.
Scenarios that passed on all three runs.
Cascade, GPT-4.1, 32.9 points higher
Calls that passed every check.
Cascade, GPT-4.1, 33.7 points higher
Calls where the agent's replies reached the expected outcome.
Cascade, GPT-4.1, 5.3 points higher
Calls where everything the agent saved matched exactly.
Cascade, GPT-4.1, 34.2 points higher
Calls where the agent went silent for 10 seconds or more.
Cascade, GPT-4.1, 0.4 points lower
How well it handled being talked over, 0 to 5.
Nova 2 Sonic, 0.03 higher
Median time to start replying after the caller finishes.
Cascade, GPT-4.1, 0.01s sooner
One minute of call at the vendor's published rates.
Not comparable
The small figure is the place among ranked models. The cascade is not ranked.
Cascade, GPT-4.1 2.17s, Nova 2 Sonic 2.18s at the median.
Amazon Nova 2 Sonic passed 41 of 82 scenarios on all three runs, and Flux → GPT-4.1 → ElevenLabs Flash passed 68. Flux → GPT-4.1 → ElevenLabs Flash is ahead, by 32.9 points.
Amazon Nova 2 Sonic saved everything exactly right on 59.9% of the calls that save data, and Flux → GPT-4.1 → ElevenLabs Flash on 94.1%. On Appointments, 82.5% against 98.8%. On Medicare, 1.5% against 81.8%. A call counts only when nothing is missing, wrong, extra or saved twice.
Flux → GPT-4.1 → ElevenLabs Flash starts replying 0.01s sooner at the median. Amazon Nova 2 Sonic takes 2.18s at the median and 2.76s at p90; Flux → GPT-4.1 → ElevenLabs Flash takes 2.17s and 2.51s. Both are measured on the call audio, from the caller finishing to the agent starting to reply.
There is no verified per-minute rate for Amazon Nova 2 Sonic and Flux → GPT-4.1 → ElevenLabs Flash, so cost cannot be compared yet.
Both ran as the whole agent in the same open-source Pipecat pipeline, with the same prompt, tools and phone line. A simulated caller worked through all 82 scenarios on live calls, three times each, 2,214 calls across the benchmark. A call passes only when the agent said the right things and saved the right data. The cascade runs the same agent for reference and is never ranked.
