- Reliability
- 50.0%
- Response time
- 2.18s
- Cost / min
- n/a
PolyAI Dialog-RSN-1 is ahead on reliability, success rate, data accuracy, calls where the agent went quiet and response time, and Amazon Nova 2 Sonic on agent responses and interruption.
Last updated Methodology by Dileep Chagam
Scenarios the agent passed on every one of its 3 runs.
Each model on its own: Amazon Nova 2 Sonic and PolyAI Dialog-RSN-1
PolyAI Dialog-RSN-1 is more reliable, by 14.6 points. Amazon Nova 2 Sonic passed 41 of 82 scenarios on all three runs, and PolyAI Dialog-RSN-1 passed 53. A scenario only counts when every run of it passed.
PolyAI Dialog-RSN-1 is more reliable, by 14.6 points. Amazon Nova 2 Sonic passed 41 of 82 scenarios on all three runs, and PolyAI Dialog-RSN-1 passed 53. A scenario only counts when every run of it passed.
For a voice agent, PolyAI Dialog-RSN-1, since it gets more scenarios right on every run. PolyAI Dialog-RSN-1 is ahead on reliability, success rate, data accuracy, calls where the agent went quiet and response time, and Amazon Nova 2 Sonic on agent responses and interruption.
PolyAI Dialog-RSN-1, by 13.1 points. Amazon Nova 2 Sonic saved everything exactly right on 59.9% of the calls that save data, and PolyAI Dialog-RSN-1 on 73.0%. On Booking calls, 82.5% against 97.7%. On Medicare calls, 1.5% against 9.1%. A call counts only when nothing is missing, wrong, extra or saved twice.
No. PolyAI Dialog-RSN-1 starts replying 0.82s sooner at the median. Amazon Nova 2 Sonic takes 2.18s at the median and 2.76s at p90; PolyAI Dialog-RSN-1 takes 1.36s and 1.87s. Both are measured on the call audio, from the caller finishing to the agent starting to reply.
Amazon Nova 2 Sonic, by 0.14 on a 0 to 5 scale. Amazon Nova 2 Sonic scores 4.99 and PolyAI Dialog-RSN-1 4.85 for how well the agent handled being talked over.
PolyAI Dialog-RSN-1, by 1.2 points. The agent went quiet for 10 seconds or more on 1.2% of Amazon Nova 2 Sonic calls and 0.0% of PolyAI Dialog-RSN-1 calls.
There is no verified per-minute rate for Amazon Nova 2 Sonic, so cost cannot be compared yet.
Cekura ran both as the whole agent in the same open-source Pipecat pipeline, with the same prompt, tools and connection. A simulated caller worked through all 82 scenarios on live calls, three times each, 3,936 calls across the benchmark. A call passes only when the agent said the right things and saved the right data.
Cekura runs these simulated calls on your agent and scores every one.