- Reliability
- 62.2%
- Response time
- 2.88s
- Cost / min
- $0.0703
Gemini 3.1 Flash Live Preview is ahead on success rate, agent responses, data accuracy and interruption, and PolyAI Dialog-RSN-1 on reliability, calls where the agent went quiet, response time and cost.
Last updated Methodology by Dileep Chagam
Scenarios the agent passed on every one of its 3 runs.
Each model on its own: Gemini 3.1 Flash Live Preview and PolyAI Dialog-RSN-1
PolyAI Dialog-RSN-1 is more reliable, by 2.4 points. Gemini 3.1 Flash Live Preview passed 51 of 82 scenarios on all three runs, and PolyAI Dialog-RSN-1 passed 53. A scenario only counts when every run of it passed.
PolyAI Dialog-RSN-1 is more reliable, by 2.4 points. Gemini 3.1 Flash Live Preview passed 51 of 82 scenarios on all three runs, and PolyAI Dialog-RSN-1 passed 53. A scenario only counts when every run of it passed.
For a voice agent, PolyAI Dialog-RSN-1, since it gets more scenarios right on every run. Gemini 3.1 Flash Live Preview is ahead on success rate, agent responses, data accuracy and interruption, and PolyAI Dialog-RSN-1 on reliability, calls where the agent went quiet, response time and cost.
Gemini 3.1 Flash Live Preview, by 6.2 points. Gemini 3.1 Flash Live Preview saved everything exactly right on 79.2% of the calls that save data, and PolyAI Dialog-RSN-1 on 73.0%. On Booking calls, 96.5% against 97.7%. On Medicare calls, 34.8% against 9.1%. A call counts only when nothing is missing, wrong, extra or saved twice.
No. PolyAI Dialog-RSN-1 starts replying 1.52s sooner at the median. Gemini 3.1 Flash Live Preview takes 2.88s at the median and 4.00s at p90; PolyAI Dialog-RSN-1 takes 1.36s and 1.87s. Both are measured on the call audio, from the caller finishing to the agent starting to reply.
Gemini 3.1 Flash Live Preview, by 0.13 on a 0 to 5 scale. Gemini 3.1 Flash Live Preview scores 4.98 and PolyAI Dialog-RSN-1 4.85 for how well the agent handled being talked over.
PolyAI Dialog-RSN-1, by 3.3 points. The agent went quiet for 10 seconds or more on 3.3% of Gemini 3.1 Flash Live Preview calls and 0.0% of PolyAI Dialog-RSN-1 calls.
PolyAI Dialog-RSN-1 is $0.0297 cheaper a minute. Gemini 3.1 Flash Live Preview costs $0.0703 a minute of call and PolyAI Dialog-RSN-1 $0.0406, at each vendor's published rates.
Cekura ran both as the whole agent in the same open-source Pipecat pipeline, with the same prompt, tools and connection. A simulated caller worked through all 82 scenarios on live calls, three times each, 3,936 calls across the benchmark. A call passes only when the agent said the right things and saved the right data.
Cekura runs these simulated calls on your agent and scores every one.