Phonic v1 vs PolyAI Dialog-RSN-1

Compare
Phonic v1Phonic
Wins 5 of 8
Reliability
90.2%
Response time
1.63s
Cost / min
$0.0600
PolyAI Dialog-RSN-1PolyAI
Wins 2 of 8
Reliability
64.6%
Response time
1.36s
Cost / min
$0.0406

Phonic v1 is ahead on reliability, success rate, agent responses, data accuracy and interruption, and PolyAI Dialog-RSN-1 on response time and cost. The two tie on calls where the agent went quiet.

Last updated Methodology by Dileep Chagam

Who is better at what

Phonic v1PolyAI Dialog-RSN-1
  • Reliability90.2%64.6%
  • Success rate95.1%67.9%
  • Agent responses97.2%93.1%
  • Data accuracy97.5%73.0%
  • Went quiet0.0%0.0%
  • Interruption4.884.85
  • Response time1.63s1.36s
  • Cost / min$0.0600$0.0406

Reliability on booking and Medicare calls

Scenarios the agent passed on every one of its 3 runs.

Booking callsMedicare calls
  1. 97%
    74%
    Phonic v1
  2. 86%
    9%
    Dialog-RSN-1

Phonic v1 vs Dialog-RSN-1 FAQ

  • Phonic v1 is more reliable, by 25.6 points. Phonic v1 passed 74 of 82 scenarios on all three runs, and PolyAI Dialog-RSN-1 passed 53. A scenario only counts when every run of it passed.

  • For a voice agent, Phonic v1, since it gets more scenarios right on every run. Phonic v1 is ahead on reliability, success rate, agent responses, data accuracy and interruption, and PolyAI Dialog-RSN-1 on response time and cost. The two tie on calls where the agent went quiet.

  • Phonic v1, by 24.5 points. Phonic v1 saved everything exactly right on 97.5% of the calls that save data, and PolyAI Dialog-RSN-1 on 73.0%. On Booking calls, 100.0% against 97.7%. On Medicare calls, 90.9% against 9.1%. A call counts only when nothing is missing, wrong, extra or saved twice.

  • No. PolyAI Dialog-RSN-1 starts replying 0.27s sooner at the median. Phonic v1 takes 1.63s at the median and 1.81s at p90; PolyAI Dialog-RSN-1 takes 1.36s and 1.87s. Both are measured on the call audio, from the caller finishing to the agent starting to reply.

  • Phonic v1, by 0.03 on a 0 to 5 scale. Phonic v1 scores 4.88 and PolyAI Dialog-RSN-1 4.85 for how well the agent handled being talked over.

  • They go quiet on the same share of calls. The agent went quiet for 10 seconds or more on 0.0% of Phonic v1 calls and 0.0% of PolyAI Dialog-RSN-1 calls.

  • PolyAI Dialog-RSN-1 is $0.0194 cheaper a minute. Phonic v1 costs $0.0600 a minute of call and PolyAI Dialog-RSN-1 $0.0406, at each vendor's published rates.

  • Cekura ran both as the whole agent in the same open-source Pipecat pipeline, with the same prompt, tools and connection. A simulated caller worked through all 82 scenarios on live calls, three times each, 3,936 calls across the benchmark. A call passes only when the agent said the right things and saved the right data.

Test your voice agent the same way.

Cekura runs these simulated calls on your agent and scores every one.