- Reliability
- 90.2%
- Response time
- 1.63s
- Cost / min
- $0.0600
Phonic v1 is ahead on reliability, success rate, agent responses, data accuracy and interruption, and PolyAI Dialog-RSN-1 on response time and cost. The two tie on calls where the agent went quiet.
Last updated Methodology by Dileep Chagam
Scenarios the agent passed on every one of its 3 runs.
Phonic v1 is more reliable, by 25.6 points. Phonic v1 passed 74 of 82 scenarios on all three runs, and PolyAI Dialog-RSN-1 passed 53. A scenario only counts when every run of it passed.
Phonic v1 is more reliable, by 25.6 points. Phonic v1 passed 74 of 82 scenarios on all three runs, and PolyAI Dialog-RSN-1 passed 53. A scenario only counts when every run of it passed.
For a voice agent, Phonic v1, since it gets more scenarios right on every run. Phonic v1 is ahead on reliability, success rate, agent responses, data accuracy and interruption, and PolyAI Dialog-RSN-1 on response time and cost. The two tie on calls where the agent went quiet.
Phonic v1, by 24.5 points. Phonic v1 saved everything exactly right on 97.5% of the calls that save data, and PolyAI Dialog-RSN-1 on 73.0%. On Booking calls, 100.0% against 97.7%. On Medicare calls, 90.9% against 9.1%. A call counts only when nothing is missing, wrong, extra or saved twice.
No. PolyAI Dialog-RSN-1 starts replying 0.27s sooner at the median. Phonic v1 takes 1.63s at the median and 1.81s at p90; PolyAI Dialog-RSN-1 takes 1.36s and 1.87s. Both are measured on the call audio, from the caller finishing to the agent starting to reply.
Phonic v1, by 0.03 on a 0 to 5 scale. Phonic v1 scores 4.88 and PolyAI Dialog-RSN-1 4.85 for how well the agent handled being talked over.
They go quiet on the same share of calls. The agent went quiet for 10 seconds or more on 0.0% of Phonic v1 calls and 0.0% of PolyAI Dialog-RSN-1 calls.
PolyAI Dialog-RSN-1 is $0.0194 cheaper a minute. Phonic v1 costs $0.0600 a minute of call and PolyAI Dialog-RSN-1 $0.0406, at each vendor's published rates.
Cekura ran both as the whole agent in the same open-source Pipecat pipeline, with the same prompt, tools and connection. A simulated caller worked through all 82 scenarios on live calls, three times each, 3,936 calls across the benchmark. A call passes only when the agent said the right things and saved the right data.
Cekura runs these simulated calls on your agent and scores every one.