Speech-to-Speech Benchmark: Real Human Calls
12 speech-to-speech models on live calls with recorded human callers.
Last updated Methodology by Prashansa Dadu
Leaderboard
Language
Industry
#ModelELO ratingEvery real conversation is a match between two models, and the one that handled the user better wins. Win more matches and the rating climbs. An average model sits at 1000.Pass³Each user was played to every model three times. Pass³ is how often a model handled the same user well all three times, so it shows who is dependable, not just lucky once.Reply timeHow long the user waits, after they stop talking, before the agent starts to answer. This is the wait in a typical conversation. Shorter feels more natural.
- ELO Rating
- 1154
- Hearing
- 75
- Turn Taking
- 81
- Speaking
- 87
- Task Handling
- 80
- Task Completion Rate
- 34.8%
- Reliability (Pass³)
- 42.9%
- Latency P50
- 1.60s
- Cost per Minute (USD)
- $0.080
What the Conversations Showed- Strongest family: Speaking (87). Weakest: Hearing (75).
- Biggest strength: Tool-Latency Handling: 89% (+31 pts vs average).
- Biggest gap: Cancellation Handling: 48% (-6 pts vs average).
- Task completion rate: 34.8%.
More Figures- Users Passed All 3 Runs
- 30 of 70
- Pass Rate (score ≥ 80)
- 65.2%
- P90 Latency
- 1.94s
- Success, Banking conversations
- 66.7%
- Success, Healthcare conversations
- 63.8%
Setup- Model
- grok-voice-3.0
- Setting
- Reasoning high
- Voice
- eve
- Turn Taking
- Service turn detection
- ELO Rating
- 1069
- Hearing
- 71
- Turn Taking
- 72
- Speaking
- 83
- Task Handling
- 82
- Task Completion Rate
- 36.2%
- Reliability (Pass³)
- 32.9%
- Latency P50
- 2.27s
- Cost per Minute (USD)
- $0.055
What the Conversations Showed- Strongest family: Speaking (83). Weakest: Hearing (71).
- Biggest strength: Entity Extraction Accuracy: 76% (+40 pts vs average).
- Biggest gap: Prosodic Cue Understanding: 38% (-27 pts vs average).
- Task completion rate: 36.2%.
More Figures- Users Passed All 3 Runs
- 23 of 70
- Pass Rate (score ≥ 80)
- 58.6%
- P90 Latency
- 2.72s
- Success, Banking conversations
- 52.4%
- Success, Healthcare conversations
- 64.8%
Setup- Model
- gpt-live-1
- Setting
- Sol on low thinking
- Voice
- marin
- Turn Taking
- Service turn detection
- ELO Rating
- 1062
- Hearing
- 68
- Turn Taking
- 79
- Speaking
- 80
- Task Handling
- 77
- Task Completion Rate
- 24.8%
- Reliability (Pass³)
- 34.3%
- Latency P50
- 3.06s
- Cost per Minute (USD)
- $0.070
What the Conversations Showed- Strongest family: Speaking (80). Weakest: Hearing (68).
- Biggest strength: Cancellation Handling: 84% (+30 pts vs average).
- Biggest gap: Tool-Latency Handling: 48% (-9 pts vs average).
- Task completion rate: 24.8%.
More Figures- Users Passed All 3 Runs
- 24 of 70
- Pass Rate (score ≥ 80)
- 58.6%
- P90 Latency
- 3.86s
- Success, Banking conversations
- 56.2%
- Success, Healthcare conversations
- 61.0%
Setup- Model
- models/gemini-3.1-flash-live-preview
- Setting
- Thinking level high
- Voice
- Charon
- Turn Taking
- Shared local turn detection
- ELO Rating
- 1054
- Hearing
- 72
- Turn Taking
- 75
- Speaking
- 79
- Task Handling
- 78
- Task Completion Rate
- 34.3%
- Reliability (Pass³)
- 32.9%
- Latency P50
- 2.16s
- Cost per Minute (USD)
- $0.085
What the Conversations Showed- Strongest family: Speaking (78). Weakest: Hearing (72).
- Biggest strength: Prosodic Cue Understanding: 100% (+35 pts vs average).
- Biggest gap: Endpointing: 47% (-13 pts vs average).
- Task completion rate: 34.3%.
More Figures- Users Passed All 3 Runs
- 23 of 70
- Pass Rate (score ≥ 80)
- 56.7%
- P90 Latency
- 2.70s
- Success, Banking conversations
- 51.4%
- Success, Healthcare conversations
- 61.9%
Setup- Model
- gpt-realtime-2.1
- Setting
- Reasoning high
- Voice
- marin
- Turn Taking
- Service turn detection
- ELO Rating
- 1030
- Hearing
- 66
- Turn Taking
- 78
- Speaking
- 82
- Task Handling
- 70
- Task Completion Rate
- 22.9%
- Reliability (Pass³)
- 28.6%
- Latency P50
- 1.44s
- Cost per Minute (USD)
- $0.080
What the Conversations Showed- Strongest family: Speaking (82). Weakest: Hearing (66).
- Biggest strength: Tool-Latency Handling: 92% (+34 pts vs average).
- Biggest gap: Premature-Response Avoidance: 19% (-20 pts vs average).
- Task completion rate: 22.9%.
More Figures- Users Passed All 3 Runs
- 20 of 70
- Pass Rate (score ≥ 80)
- 51.9%
- P90 Latency
- 1.83s
- Success, Banking conversations
- 49.5%
- Success, Healthcare conversations
- 54.3%
Setup- Model
- grok-voice-think-fast-2.0
- Setting
- Reasoning high
- Voice
- eve
- Turn Taking
- Service turn detection
- ELO Rating
- 1030
- Hearing
- 67
- Turn Taking
- 76
- Speaking
- 70
- Task Handling
- 79
- Task Completion Rate
- 24.8%
- Reliability (Pass³)
- 31.4%
- Latency P50
- 1.99s
- Cost per Minute (USD)
- Not published
What the Conversations Showed- Strongest family: Task Handling (79). Weakest: Hearing (67).
- Biggest strength: Prosodic Cue Understanding: 87% (+22 pts vs average).
- Biggest gap: Language Switching: 32% (-31 pts vs average).
- Task completion rate: 24.8%.
More Figures- Users Passed All 3 Runs
- 22 of 70
- Pass Rate (score ≥ 80)
- 52.9%
- P90 Latency
- 2.32s
- Success, Banking conversations
- 56.2%
- Success, Healthcare conversations
- 49.5%
Setup- Model
- phonic-1
- Setting
- —
- Voice
- —
- Turn Taking
- Service turn detection
- ELO Rating
- 1017
- Hearing
- 67
- Turn Taking
- 69
- Speaking
- 77
- Task Handling
- 78
- Task Completion Rate
- 25.7%
- Reliability (Pass³)
- 37.1%
- Latency P50
- 1.86s
- Cost per Minute (USD)
- $0.080
What the Conversations Showed- Strongest family: Task Handling (78). Weakest: Hearing (66).
- Biggest strength: Tool-Latency Handling: 83% (+25 pts vs average).
- Biggest gap: Endpointing: 15% (-45 pts vs average).
- Task completion rate: 25.7%.
More Figures- Users Passed All 3 Runs
- 26 of 70
- Pass Rate (score ≥ 80)
- 54.8%
- P90 Latency
- 2.10s
- Success, Banking conversations
- 49.5%
- Success, Healthcare conversations
- 60.0%
Setup- Model
- azure-realtime
- Setting
- No reasoning setting
- Voice
- ava
- Turn Taking
- Service turn detection
- ELO Rating
- 1016
- Hearing
- 54
- Turn Taking
- 79
- Speaking
- 80
- Task Handling
- 67
- Task Completion Rate
- 22.9%
- Reliability (Pass³)
- 24.3%
- Latency P50
- 2.10s
- Cost per Minute (USD)
- Not published
What the Conversations Showed- Strongest family: Speaking (80). Weakest: Hearing (54).
- Biggest strength: Backchannel vs Barge-in Detection: 100% (+12 pts vs average).
- Biggest gap: Prosodic Cue Understanding: 40% (-25 pts vs average).
- Task completion rate: 22.9%.
More Figures- Users Passed All 3 Runs
- 17 of 70
- Pass Rate (score ≥ 80)
- 53.3%
- P90 Latency
- 2.60s
- Success, Banking conversations
- 50.5%
- Success, Healthcare conversations
- 56.2%
Setup- Model
- gemini-3.8-live-extended-thinking
- Setting
- Thinking level high
- Voice
- Charon
- Turn Taking
- Service turn detection
- ELO Rating
- 968
- Hearing
- 56
- Turn Taking
- 72
- Speaking
- 82
- Task Handling
- 61
- Task Completion Rate
- 16.2%
- Reliability (Pass³)
- 27.1%
- Latency P50
- 2.01s
- Cost per Minute (USD)
- Not published
What the Conversations Showed- Strongest family: Speaking (82). Weakest: Hearing (56).
- Biggest strength: Waiting Through Hold: 89% (+12 pts vs average).
- Biggest gap: Correction Handling: 33% (-30 pts vs average).
- Task completion rate: 16.2%.
More Figures- Users Passed All 3 Runs
- 19 of 70
- Pass Rate (score ≥ 80)
- 46.2%
- P90 Latency
- 2.52s
- Success, Banking conversations
- 37.1%
- Success, Healthcare conversations
- 55.2%
Setup- Model
- gemini-3.8-live
- Setting
- No thinking level
- Voice
- Charon
- Turn Taking
- Service turn detection
- ELO Rating
- 921
- Hearing
- 54
- Turn Taking
- 74
- Speaking
- 71
- Task Handling
- 65
- Task Completion Rate
- 13.8%
- Reliability (Pass³)
- 14.3%
- Latency P50
- 2.40s
- Cost per Minute (USD)
- $0.020
What the Conversations Showed- Strongest family: Turn Taking (74). Weakest: Hearing (54).
- Biggest strength: Backchannel vs Barge-in Detection: 100% (+12 pts vs average).
- Biggest gap: Entity Extraction Accuracy: 10% (-26 pts vs average).
- Task completion rate: 13.8%.
More Figures- Users Passed All 3 Runs
- 10 of 70
- Pass Rate (score ≥ 80)
- 39.5%
- P90 Latency
- 2.88s
- Success, Banking conversations
- 34.3%
- Success, Healthcare conversations
- 44.8%
Setup- Model
- gpt-realtime-2.1-mini
- Setting
- Reasoning high
- Voice
- marin
- Turn Taking
- Service turn detection
- ELO Rating
- 847
- Hearing
- 60
- Turn Taking
- 46
- Speaking
- 71
- Task Handling
- 54
- Task Completion Rate
- 12.4%
- Reliability (Pass³)
- 12.9%
- Latency P50
- 2.46s
- Cost per Minute (USD)
- $0.112
What the Conversations Showed- Strongest family: Speaking (71). Weakest: Turn Taking (46).
- Biggest strength: Waiting Through Hold: 88% (+11 pts vs average).
- Biggest gap: Backchannel vs Barge-in Detection: 6% (-82 pts vs average).
- Task completion rate: 12.4%.
More Figures- Users Passed All 3 Runs
- 9 of 70
- Pass Rate (score ≥ 80)
- 31.4%
- P90 Latency
- 3.16s
- Success, Banking conversations
- 21.9%
- Success, Healthcare conversations
- 41.0%
Setup- Model
- opal
- Setting
- No reasoning setting
- Voice
- ElevenLabs 21m00Tcm4TlvDq8ikWAM
- Turn Taking
- Service turn detection
- ELO Rating
- 832
- Hearing
- 46
- Turn Taking
- 75
- Speaking
- 66
- Task Handling
- 54
- Task Completion Rate
- 13.8%
- Reliability (Pass³)
- 11.4%
- Latency P50
- 2.76s
- Cost per Minute (USD)
- Not published
What the Conversations Showed- Strongest family: Turn Taking (75). Weakest: Hearing (46).
- Biggest strength: Endpointing: 90% (+31 pts vs average).
- Biggest gap: Tool Error Handling: 30% (-55 pts vs average).
- Task completion rate: 13.8%.
More Figures- Users Passed All 3 Runs
- 8 of 70
- Pass Rate (score ≥ 80)
- 27.1%
- P90 Latency
- 3.40s
- Success, Banking conversations
- 20.0%
- Success, Healthcare conversations
- 34.3%
Setup- Model
- amazon.nova-2-sonic-v1:0
- Setting
- Endpointing medium
- Voice
- matthew
- Turn Taking
- Shared local turn detection
Reference only, not ranked: a speech-to-text, LLM and text-to-speech pipeline on the same conversations. Its ELO is on the same scale and does not change any model's rating.
- ELO Rating
- 955
- Hearing
- 53
- Turn Taking
- 73
- Speaking
- 73
- Task Handling
- 65
- Task Completion Rate
- 12.4%
- Reliability (Pass³)
- 25.7%
- Latency P50
- 2.06s
- Cost per Minute (USD)
- Not published
What the Conversations Showed- Strongest family: Turn Taking (73). Weakest: Hearing (53).
- Biggest strength: Endpointing: 74% (+15 pts vs average).
- Biggest gap: Prosodic Cue Understanding: 7% (-58 pts vs average).
- Task completion rate: 12.4%.
More Figures- Users Passed All 3 Runs
- 18 of 70
- Pass Rate (score ≥ 80)
- 44.8%
- P90 Latency
- 2.49s
- Success, Banking conversations
- 45.7%
- Success, Healthcare conversations
- 43.8%
Setup- Model
- flux-general-en (English callers) or nova-3 multi (callers who switch language) → gpt-4.1 → eleven_flash_v2_5
- Setting
- No reasoning setting
- Voice
- ElevenLabs 21m00Tcm4TlvDq8ikWAM
- Turn Taking
- Speech-to-text turn detection
Click a column to rank by it. Open a model for its skill scores and notes from the conversations.
How to read this table
- ELO · Head-to-head rating
- On every real human call, each pair of models is compared and the one that handled the caller better wins. 1000 is the average model; 100 points higher wins about 64% of calls.
- Pass³ · Handled well on all 3 runs
- Share of real callers the model handled well (80+ of 100) on every one of its 3 runs of the same call.
- Skill families · Hearing, Turn Taking, Task Handling and Speaking
- 25 skills in 4 families, each scored 0–100. Task Handling is half the task skills and half how much of the caller's task got done.
- Task Completion Rate · Every step done
- Share of calls where every step the caller needed was done.
- Latency P50 · Median reply time
- Median time from the caller finishing to the agent starting to reply. Lower is better.
- Cost/Min (USD) · Published price
- One minute of call at the vendor's published rates. Blank where no rate is published.
Best Rated, Fastest and Cheapest
Best rated
Grok Voice 3.0
1154 ELOELO ratingEvery real conversation is a match between two models, and the one that handled the user better wins. Win more matches and the rating climbs. An average model sits at 1000.
- 2GPT-Live 1 + GPT-6 Sol1069
- 3Gemini 3.1 Flash Live Preview1062
- 4GPT Realtime 2.11054
- 5Grok Voice Think Fast 2.01030
Fastest reply
Grok Voice Think Fast 2.0
1.44s median waitReply timeHow long the user waits, after they stop talking, before the agent starts to answer. This is the wait in a typical conversation. Shorter feels more natural.
- 2Grok Voice 3.01.60s
- 3Azure Realtime1.86s
- 4Phonic-11.99s
- 5Gemini 3.8 Live2.01s
Lowest price
GPT Realtime 2.1 Mini
$0.020 per minuteCost per minuteWhat the provider charges for one minute of audio, at its published price. A dash means the provider has not published one.
- 2GPT-Live 1 + GPT-6 Sol$0.055
- 3Gemini 3.1 Flash Live Preview$0.070
- 4Azure Realtime$0.080
- 5Grok Voice Think Fast 2.0$0.080
Results by Call
How often each model handled real banking and healthcare callers well, and how often it did so in all three runs.
Banking callsHealthcare calls
- Grok Voice 3.0
- Banking calls
- 40.0%
- Healthcare calls
- 45.7%
- Gap
- 5.7 points
40%46%Grok Voice 3.0 - Azure Realtime
- Banking calls
- 31.4%
- Healthcare calls
- 42.9%
- Gap
- 11.4 points
31%43%Azure Realtime - GPT Realtime 2.1
- Banking calls
- 25.7%
- Healthcare calls
- 40.0%
- Gap
- 14.3 points
26%40%GPT Realtime 2.1 - Gemini 3.1 Flash Live Preview
- Banking calls
- 28.6%
- Healthcare calls
- 40.0%
- Gap
- 11.4 points
29%40%Gemini 3.1 Flash Live Preview - Gemini 3.8 Live
- Banking calls
- 17.1%
- Healthcare calls
- 37.1%
- Gap
- 20.0 points
17%37%Gemini 3.8 Live - GPT-Live 1 + GPT-6 Sol
- Banking calls
- 28.6%
- Healthcare calls
- 37.1%
- Gap
- 8.6 points
29%37%GPT-Live 1 + GPT-6 Sol - Grok Voice Think Fast 2.0
- Banking calls
- 22.9%
- Healthcare calls
- 34.3%
- Gap
- 11.4 points
23%34%Grok Voice Think Fast 2.0 - Phonic-1
- Banking calls
- 31.4%
- Healthcare calls
- 31.4%
- Gap
- 0.0 points
31%31%Phonic-1 - Gemini 3.8 Live Extended Thinking
- Banking calls
- 20.0%
- Healthcare calls
- 28.6%
- Gap
- 8.6 points
20%29%Gemini 3.8 Live Extended Thinking - CascadeNot ranked
- Banking calls
- 25.7%
- Healthcare calls
- 25.7%
- Gap
- 0.0 points
26%26%Cascade - GPT Realtime 2.1 Mini
- Banking calls
- 8.6%
- Healthcare calls
- 20.0%
- Gap
- 11.4 points
9%20%GPT Realtime 2.1 Mini - Amazon Nova 2 Sonic
- Banking calls
- 5.7%
- Healthcare calls
- 17.1%
- Gap
- 11.4 points
6%17%Amazon Nova 2 Sonic - Deepslate Opal
- Banking calls
- 8.6%
- Healthcare calls
- 17.1%
- Gap
- 8.6 points
9%17%Deepslate Opal
ModelBankingHealth
- Grok Voice 3.04046
- Azure Realtime3143
- Gemini 3.1 Flash Live Preview2940
- GPT Realtime 2.12640
- GPT-Live 1 + GPT-6 Sol2937
- Phonic-13131
- Grok Voice Think Fast 2.02334
- Gemini 3.8 Live1737
- Cascade2626
- Gemini 3.8 Live Extended Thinking2029
- GPT Realtime 2.1 Mini920
- Deepslate Opal917
- Amazon Nova 2 Sonic617
6%46%Yellow is weaker, green is stronger
Who Does Each Skill Best
Hearing the caller
A caller reads out a card number on a crackly line while the TV plays behind them. Hearing is whether the agent catches what was really said, notices how it was said, and asks again when it missed something.
Who does it bestScore out of 100Score out of 100How often the model got this kind of moment right, across every conversation where it came up. Higher is better.
- 01Grok Voice 3.075
- 02GPT Realtime 2.172
- 03GPT-Live 1 + GPT-6 Sol71
- 04Gemini 3.1 Flash Live Preview68
- 05Phonic-167
- 06Azure Realtime67
- 07Grok Voice Think Fast 2.066
- 08Deepslate Opal60
- 09Gemini 3.8 Live56
- 10Gemini 3.8 Live Extended Thinking54
- 11GPT Realtime 2.1 Mini54
- 12Amazon Nova 2 Sonic46
Listen: a good and a bad call
Pick a skill to hear a real call where a model got it right, and one where it did not.
Where Each Model Is Strong or Weak
Four skill families, each scored 0–100. A model can lead on one and trail on another. Pick two models to compare their shapes.
Tap two models
Grok Voice 3.0Gemini 3.1 Flash Live Preview
Grok Voice 3.0: strongest at speaking (87), weakest at hearing (75). Top of all models on hearing, turn taking and speaking.
Gemini 3.1 Flash Live Preview: strongest at speaking (80), weakest at hearing (68).
Each axis is a family score from 0 (centre) to 100 (outer ring), so a bigger shape is a stronger model. Hover a dot for the exact value.
Score by Call Type
Each row is a kind of situation real callers create. The same model can handle noisy lines well and still trip when a caller changes their mind.
Grok Voice 3.0 is above average on 10 of 10 call typesAverage model
- Noisy / Degraded Audio62%+13
- Compliance & Handoff69%+13
- Corrections & Changes67%+7
- Entity-Heavy74%+14
- Language Switching83%+21
- Pauses & Holds73%+6
- Multi-Speaker74%+5
- Tool Latency & Errors91%+19
- Affective Speech92%+14
- Barge-in & Overlap98%+6
Average pass rate of the skills each call type tests, every skill counted once, across all 3 runs. Hardest call types first.
Put Two Models Head to Head
Pick any two models and see them side by side. Every pair has its own page with the full numbers.
MeasureGrok Voice 3.0GPT-Live 1 + GPT-6 Sol
- ELO Rating11541069
- Reliability (Pass³)42.9%32.9%
- Hearing7571
- Turn Taking8172
- Speaking8783
- Task Handling8082
- Task Completion Rate34.8%36.2%
- Latency P501.60s2.27s
- Cost per Minute (USD)$0.080$0.055
- Measures won6 of 93 of 9
Banking and healthcare calls together. The check marks the better value on each measure.All 78 comparisons
The Top-Rated Model Against the Next Six
How We Tested
Every model took the same recorded human callers on live calls, ran the same agent and was scored the same way.
- Step 1Recorded Human CallersCallers are recordings of real people. Each model hears exactly the same recordings.
- Step 2Live Calls, three Runs EachEach model takes all 70 calls, banking and healthcare, three times. Every call counts.
- Step 3Only the Model ChangesSame agent, prompt, tools and phone connection for every model. Only the speech-to-speech model is swapped.
- Step 4Graded Skill by SkillEach call is graded on 25 skills in 4 families: Hearing, Turn Taking, Task Handling and Speaking. A call passes at a score of 80 out of 100.
Real Human Calls FAQ
Which speech-to-speech model handles real callers best?
On human calls, Grok Voice 3.0 has the highest ELO rating (1154), winning the most head-to-head calls. Grok Voice 3.0 is the most reliable, handling 42.9% of callers well on all three runs. Grok Voice Think Fast 2.0 replies fastest, at 1.44s median. GPT Realtime 2.1 Mini is the cheapest, at $0.020 a minute of call. The right pick depends on which of the three your voice agent needs most, and all three are measured on the same live calls.
On human calls, Grok Voice 3.0 has the highest ELO rating (1154), winning the most head-to-head calls. Grok Voice 3.0 is the most reliable, handling 42.9% of callers well on all three runs. Grok Voice Think Fast 2.0 replies fastest, at 1.44s median. GPT Realtime 2.1 Mini is the cheapest, at $0.020 a minute of call. The right pick depends on which of the three your voice agent needs most, and all three are measured on the same live calls.
The callers are recordings of real people speaking, with the noise, hesitation, interruptions and accents of real phone calls. Each model hears exactly the same recordings.
By ELO rating. On every call, each pair of models is compared and the one that handled the real caller better wins; 1000 is the average model. Next to it, Pass³ shows reliability: the share of callers a model handled well (80+ of 100) in all three runs.
Yes. Every model runs the same agent, prompt, tools, phone connection, callers and scoring. Only the speech-to-speech model changes.
The cascade (speech-to-text, a text model and text-to-speech) is a reference point and is not ranked. It uses English speech-to-text for callers who speak English throughout and multilingual speech-to-text for callers who switch language. It has an ELO on the same scale, rated against the ranked models without changing any of their ratings.
Test your voice agent the same way.
Cekura runs these simulated calls on your agent and scores every one.