Which configuration ranked first?

PolyAI ranked first at 87.80% pass^3.

Can the Cekura Bench leaderboard be sorted by a different metric?

Yes. The published table can be reordered by provider, repeatable reliability, task completion, infrastructure reliability, interruption handling, conversational naturalness, or mean main-agent response time.

How was Gemini Live configured for its rerun?

The published Gemini Live result uses high reasoning and hybrid VAD. Its completed rerun is included in the leaderboard and reliability versus response-time chart.

How many calls were retained?

246 calls per configuration, 2460 across the 10-configuration cohort.

Which PolyAI results are included?

Medicare result 960277 and Appointment result 960343 in project 15514: 246 completed calls across 82 scenarios with three repeats each, collected on October 21, 2026. Pass rates retain the reported verdicts for applicable scored rubric gates. Repetition and call termination were not scored; tool accuracy covers 237 calls and latency covers 245. The public All runs report includes both result groups.

Which Smallest AI results are included?

Appointment result 1013698 and Medicare results 1014171, 1014575 and 1014602 in project 8197: 246 completed calls across 82 scenarios with three repeats each, collected on November 3, 2026 on Atoms with the electron model. The scenarios are date-shifted copies of the same suite, with every July 2026 date moved 22 weeks to December, weekdays unchanged. Atoms tool parameters cannot be arrays, so it sends Medicare list arguments as comma-separated text; its tool accuracy and pass rates use Tool Call Accuracy (list text accepted), which reads that text as a list and is otherwise identical. With the standard metric it scores 45.12% pass^3. Repetition and call termination were not scored; tool accuracy covers 237 calls.

Bench

Book a demo

Voice agent workflow benchmark

LeaderboardSource

Reliability vs. response time

Higher and further left is better

Best quadrantBest picks

Reliability, pass³ (%)

Average response time (s)

Who wins what

The leader on each measure

Most reliablePolyAI87.8%pass³
Fastest replyElevenLabs1.27smean
Best voiceElevenLabs4.47tone, 0 to 5

Full leaderboard

10 voice agent platforms ran the same agent on the same calls. This is who got them right.

Last updated Nov 3, 2026Methodology by Luis Ojeda

Compare the metrics that matter

Task completion

Calls that fully reached the expected outcome.

PolyAI leads at 98.8%. Gemini Live is last at 87.8%.

How this is measured

A call counts as complete only when Expected Outcome receives the full 5/5 score. Coverage varies when a call has no outcome evidence.

98.8%
PolyAI
97.6%
Telnyx
97.6%
Vapi
95.1%
LiveKit
94.3%
Smallest AI
94.2%
Pipecat
93.9%
Retell
92.7%
GPT Realtime
91.5%
ElevenLabs
87.8%
Gemini Live

Scale starts at 80% so small gaps show.

Infrastructure reliability

Calls with no provider or connection issue. Failed connections still count.

ElevenLabs leads at 100.0%. Gemini Live is last at 72.4%.

How this is measured

All 246 retained calls per configuration are included. A call counts as clean only when Infrastructure Issues receives 5/5.

100.0%
ElevenLabs
100.0%
Telnyx
99.2%
PolyAI
99.2%
LiveKit
98.4%
Retell
97.2%
Pipecat
96.3%
Smallest AI
95.5%
GPT Realtime
82.9%
Vapi
72.4%
Gemini Live

Scale starts at 60% so small gaps show.

Interruption handling

How well the agent avoids talking over the caller, out of 5.

Retell leads at 5.00. Telnyx is last at 4.63.

How this is measured

Mean Interruption Score out of 5, on calls where the caller pauses, coughs or goes quiet mid-sentence.

5.00
Retell
4.99
PolyAI
4.98
GPT Realtime
4.97
LiveKit
4.97
Pipecat
4.97
Gemini Live
4.96
ElevenLabs
4.91
Smallest AI
4.73
Vapi
4.63
Telnyx

Scale starts at 4 so small gaps show.

Voice naturalness

How clear and natural the voice sounded, out of 5.

ElevenLabs leads at 4.47. Pipecat is last at 3.74.

How this is measured

Mean Voice Tone + Clarity score out of 5. Gemini Live is not shown because this evaluator was not included in its hybrid-VAD rerun.

4.47
ElevenLabs
4.37
Smallest AI
4.36
Retell
4.36
LiveKit
4.25
GPT Realtime
4.21
Telnyx
4.08
Vapi
3.97
PolyAI
3.74
Pipecat

Scale starts at 3 so small gaps show.

Leaderboard

Ranked by reliability: the share of 82 scenarios each platform passed on all three runs. Select any column to sort by it. For the wider field, see our pick of the best AI voice agent platforms.

							Runs
1PolyAI	87.8%	98.8%	99.2%	4.99	3.97	2.46s
2Retell	75.6%	93.9%	98.4%	5.00	4.36	2.21s
3LiveKit	70.7%	95.1%	99.2%	4.97	4.36	2.59s
4ElevenLabs	69.5%	91.5%	100.0%	4.96	4.47	1.27s
5Telnyx	68.3%	97.6%	100.0%	4.63	4.21	1.82s
6GPT Realtime	64.6%	92.7%	95.5%	4.98	4.25	1.58s
7Pipecat	63.4%	94.2%	97.2%	4.97	3.74	1.97s
8Vapi	59.8%	97.6%	82.9%	4.73	4.08	3.08s
9Smallest AI	56.1%	94.3%	96.3%	4.91	4.37	2.71s
10Gemini Live	30.5%	87.8%	72.4%	4.97	n/a	3.05s
–Your agent							Get scored

Repeatable reliability87.80%75.61%
Task completion98.78%93.88%
Infrastructure-clean calls99.19%98.37%
Tool call accuracy4.90/54.84/5
Voice tone and clarity3.97/54.36/5
Interruption handling4.99/55.00/5
Mean response time2.46s2.21s
Measures won4 of 73 of 7

The check marks the better value on each measure.All 45 comparisons

Experiments

View all experiments →

Experiment 02 · July 2026

Medicare workflow

The same Medicare agent deployed on six platforms. Workflow pass³ ranged from 65.2% to 95.7%.

Experiment 01 · July 2026

Telnyx + Kimi K2.6

Prompt, tools, voice, STT and TTS held fixed on Telnyx. Kimi K2.6 reached 88.1% pass³ against 76.3% for gpt-4.1.

Earlier studies: Four-provider configurations · Fixed-stack benchmark

How we tested

Every platform ran the same 82 caller scenarios, three times each, on the stack it chose. A scenario passes only when all three runs pass.

10
configurations: 82
caller scenarios: 3
runs of each scenario: 2,460
calls scored

Benchmark on GitHub

Full methodology

Step by step

1
Providers chose what to test
Cekura invited voice agent platforms to submit the configuration they wanted benchmarked. Providers chose their models, speech components, and settings.
2
Everyone received the same brief
Cekura shared the system prompt, tool definitions, test-case summaries, and test data. Transfers to other agents were the only configuration restriction.
3
Each scenario ran three times
The same 82 caller situations and evaluator suite were used for every configuration. A scenario earns pass³ only when all three retained runs pass.
4
Failures remain in the results
Calls that did not connect or produced no transcript stay in the denominator. Missing provider evidence is not removed from the release.

Data notes

PolyAI’s Medicare and Appointment results were collected on October 21, 2026 in project 15514 (results 960277 and 960343).
Repetition and call termination have no scored coverage in these results. Tool accuracy covers 237 of 246 calls and response time covers 245.
Pass rates retain the reported verdicts for applicable scored rubric gates. The public “All runs” report includes both result groups.
Smallest AI’s results were collected on November 3, 2026 in project 8197 (Appointment result 1013698; Medicare results 1014171, 1014575 and 1014602). Its agent reads the real date and refused July dates as past, so it ran date-shifted copies of the same 82 scenarios, with every July 2026 date moved 22 weeks to December and weekdays unchanged.
Smallest AI’s Atoms tool parameters cannot be arrays, so it sends the Medicare contract’s two list arguments as comma-separated text. Its tool accuracy and pass rates use Tool Call Accuracy (list text accepted), which reads that text as a list and is otherwise identical to the standard check. With the standard check it scores 45.12% pass³ (148 of 246 calls). Repetition and call termination have no scored coverage; tool accuracy covers 237 of 246 calls. Its 26 Medicare calls refused for an empty test-account balance were re-run.