- Reliability
- 62.2%
- Response time
- 2.88s
- Cost / min
- $0.070
Gemini 3.1 Flash Live Preview is ahead on reliability, success rate, agent responses and data accuracy, and Amazon Nova 2 Sonic on calls where the agent went quiet, interruption and response time.
Last updated Methodology by Dileep Chagam
Right every time, on booking and Medicare calls
Gemini 3.1 Flash Live Preview is more reliable, by 12.2 points. Gemini 3.1 Flash Live Preview passed 51 of 82 scenarios on all three runs, and Amazon Nova 2 Sonic passed 41. A scenario only counts when every run of it passed.
Gemini 3.1 Flash Live Preview is more reliable, by 12.2 points. Gemini 3.1 Flash Live Preview passed 51 of 82 scenarios on all three runs, and Amazon Nova 2 Sonic passed 41. A scenario only counts when every run of it passed.
For a voice agent, Gemini 3.1 Flash Live Preview, since it gets more scenarios right on every run. Gemini 3.1 Flash Live Preview is ahead on reliability, success rate, agent responses and data accuracy, and Amazon Nova 2 Sonic on calls where the agent went quiet, interruption and response time.
Gemini 3.1 Flash Live Preview, by 19.3 points. Gemini 3.1 Flash Live Preview saved everything exactly right on 79.2% of the calls that save data, and Amazon Nova 2 Sonic on 59.9%. On Booking calls, 96.5% against 82.5%. On Medicare calls, 34.8% against 1.5%. A call counts only when nothing is missing, wrong, extra or saved twice.
No. Amazon Nova 2 Sonic starts replying 0.70s sooner at the median. Gemini 3.1 Flash Live Preview takes 2.88s at the median and 4.00s at p90; Amazon Nova 2 Sonic takes 2.18s and 2.76s. Both are measured on the call audio, from the caller finishing to the agent starting to reply.
Amazon Nova 2 Sonic, by 0.01 on a 0 to 5 scale. Gemini 3.1 Flash Live Preview scores 4.98 and Amazon Nova 2 Sonic 4.99 for how well the agent handled being talked over.
Amazon Nova 2 Sonic, by 2.0 points. The agent went quiet for 10 seconds or more on 3.3% of Gemini 3.1 Flash Live Preview calls and 1.2% of Amazon Nova 2 Sonic calls.
There is no verified per-minute rate for Amazon Nova 2 Sonic, so cost cannot be compared yet.
Cekura ran both as the whole agent in the same open-source Pipecat pipeline, with the same prompt, tools and connection. A simulated caller worked through all 82 scenarios on live calls, three times each, 3,198 calls across the benchmark. A call passes only when the agent said the right things and saved the right data.