Converse-STT speech-to-text benchmark
A caller stops talking. Soniox STT-RT v5 has the transcript ready in 38ms. AssemblyAI Universal 3.6 Pro takes 91ms, but gets the fewest words wrong.
Last updated Methodology by Dileep Chagam
With OcularAccuracy vs speed
Each dot is one model. Bottom-left is best: fewer errors and a faster final transcript.
Who gets the words right
On the public clips, AssemblyAI Universal 3.6 Pro gets about 2 words in 100 wrong. Real calls are harder: 14 of the other 16 models land too close to the leader to call.
- Pipecat 1.77%Ocular 4.50%First 485msFinal 91ms
- Pipecat 1.93%Ocular 4.23%First 489msFinal 180ms
- Pipecat 2.00%Ocular 4.19%First 4.48sFinal 475ms*
- Pipecat 2.10%Ocular 2.93%First 2.40sFinal 235ms
- Pipecat 2.16%Ocular 3.53%First 1.38sFinal 543ms
- Pipecat 2.39%Ocular 3.09%First 1.51sFinal 123ms
- Pipecat 2.46%Ocular 3.93%First 1.19sFinal 147ms
- Pipecat 2.54%Ocular 3.92%First 1.20sFinal 38ms
- Pipecat 2.59%Ocular 3.80%First 4.50sFinal 568ms*
- Pipecat 2.72%Ocular 10.60%First 1.06sFinal 41ms
- Pipecat 3.35%Ocular 4.45%First at speech endFinal 630ms
- Pipecat 3.46%Ocular 4.39%First 1.06sFinal 103ms
- Pipecat 3.66%Ocular 3.46%First 1.06sFinal 201ms
- Pipecat 3.90%Ocular 12.45%First at speech endFinal 652ms
- Pipecat 3.90%Ocular 4.18%First 791msFinal 108ms
- Pipecat 4.96%Ocular 4.53%First 793msFinal 114ms
- Pipecat 6.48%Ocular 4.45%First 1.71sFinal 212ms
- Bars use a linear scale from zero. Lower is better for every metric.
How to read this table
- WER · Word error rate
- Share of words the model got wrong: replaced, missing or added. Lower is better.
- Pipecat Dataset
- 1,000 short public clips from Pipecat's STT benchmark dataset.
- Ocular Dataset
- 8 recordings from 4 real phone conversations, licensed from Ocular.
- Likely range · 95% range
- Where the model's true score probably falls, given how much audio was tested. Overlapping ranges are too close to call.
- TTFT · Time to first text
- How long after the audio starts streaming until the first words appear.
- TTFS · Final-text delay
- How long after the caller stops until the full transcript arrives. A voice agent waits for this before it replies.
- P50 / P90 / P95 · Typical, slow and slowest turns
- Half of turns are faster than P50. Only 1 in 10 is slower than P90, and 1 in 20 slower than P95.
Likely ranges come from resampling the Pipecat clips, or the Ocular conversations, 10,000 times and keeping the middle 95% of results. Lower is better on every column.
Accuracy and speed pull apart
Getting words right takes time. The most accurate model needs 91ms for the final transcript. The fastest needs 38ms, but gets more words wrong: 2.54% against 1.77%.
Public clips vs real calls
Each dot is one model. Bottom-left is best on both datasets.
First words vs final transcript
Each dot is one model. Bottom-left is fastest at both.
Model ranking
Word error rate on 1,000 clips from Pipecat's STT benchmark dataset. Lower is better.
| # | Model | Pipecat Dataset | clips | |
|---|---|---|---|---|
| 1 | AssemblyAI Universal 3.6 Pro | 1.77% | 1,000 | |
| 2 | AssemblyAI Universal 3.5 Pro | 1.93% | 1,000 | |
| 3 | Google Chirp 3 | 2.00% | 1,000 | |
| 4 | Reson8 | 2.10% | 1,000 | |
| 5 | GPT Realtime Whisper | 2.16% | 1,000 | |
| 6 | Cartesia Ink 2 | 2.39% | 1,000 | |
| 7 | Speechmatics Linden | 2.46% | 998 | |
| 8 | Soniox STT-RT v5 | 2.54% | 1,000 | |
| 9 | Google Chirp 2 | 2.59% | 1,000 | |
| 10 | Inworld STT-1 | 2.72% | 1,000 | |
| 11 | GPT-4o Mini Transcribe | 3.35% | 1,000 | |
| 12 | Deepgram Nova-3 | 3.46% | 1,000 | |
| 13 | Smallest Pulse | 3.66% | 1,000 | |
| 14 | GPT-4o Transcribe | 3.90% | 1,000 | |
| 15 | Deepgram Flux English | 3.90% | 999 | |
| 16 | Deepgram Flux Multilingual | 4.96% | 999 | |
| 17 | Gradium | 6.48% | 1,000 |
Counts are the clips each model returned usable text for, so they differ between models and between datasets.
Final-text timing uses each provider's supported finalization contract. Compare models with similar contracts before treating small latency gaps as meaningful.
Pick by what your agent needs
So the right model depends on what your agent can't afford to get wrong. Start there.
- Most accurate on clean clipsAssemblyAI Universal 3.6 ProPipecat WER 1.77%likely 1.55 to 2.00%, 5 others overlap
- Most accurate on real conversationsReson8Ocular WER 2.93%likely 2.29 to 3.68%, 14 others overlap
- First words on screen soonestAssemblyAI Universal 3.6 ProFirst text 485ms
- Quickest turn-takingSoniox STT-RT v5Final text 38ms
Compare any two models
Already have two in mind? Put them side by side.
- Pipecat Dataset WER
- 1.77%1st0.78 points lower
- 2.54%8th
- Ocular Dataset WER
- 4.50%14thToo close to call
- 3.92%6thToo close to call
- Time to first text
- 485ms1st710ms sooner
- 1.20s9th
- Final-text delay
- 91ms3rd
- 38ms1st53ms sooner
The most accurate model against the next six
- Universal 3.6 Pro vs Universal 3.5 Pro
- Universal 3.6 Pro vs Chirp 3
- Universal 3.6 Pro vs Reson8
- Universal 3.6 Pro vs Realtime Whisper
- Universal 3.6 Pro vs Ink 2
- Universal 3.6 Pro vs Linden
Or open one model
- 1AssemblyAI Universal 3.6 Pro
- 2AssemblyAI Universal 3.5 Pro
- 3Google Chirp 3
- 4Reson8
- 5GPT Realtime Whisper
- 6Cartesia Ink 2
- 7Speechmatics Linden
- 8Soniox STT-RT v5
- 9Google Chirp 2
- 10Inworld STT-1
- 11GPT-4o Mini Transcribe
- 12Deepgram Nova-3
- 13Smallest Pulse
- 14GPT-4o Transcribe
- 15Deepgram Flux English
- 16Deepgram Flux Multilingual
- 17Gradium
How we tested
Every model heard the same audio and was scored the same way.
Same audio
All 17 models transcribed the same 1,000 public clips and the same 4 real phone calls.
Same scoring
WER counts every word replaced, missing or added, against the reference transcript.
Same clock
Speed comes from the same 206 streamed turns, at typical, slow and worst-case (P50, P90, P95).
Where the audio comes from
Pipecat Dataset
1,000 public voice-agent clips from Pipecat's STT benchmark dataset. Its reference transcripts are generated with Gemini and described by Pipecat as human-reviewed.
View Pipecat datasetOcular Dataset
Eight licensed speaker recordings from four conversations, provided by Ocular for this benchmark.
About OcularQuestions
What is the best speech-to-text model in 2026?
AssemblyAI Universal 3.6 Pro has the lowest word error rate on the Pipecat Dataset, at 1.77%. 5 other models are within its margin of error there, so that lead is not certain. Reson8 has the lowest error rate on the Ocular conversation recordings, at 2.93%. 14 other models are within its margin of error there, so that lead is not certain. Soniox STT-RT v5 returns final text soonest, at 38ms median. Which is best depends on whether accuracy or speed matters more for your agent, so the leaderboard keeps the two apart.
What does the speech-to-text benchmark measure?
It measures transcription accuracy with word error rate, streaming responsiveness with time to first text, and finalization speed with time to final text.
What is a good word error rate?
Lower word error rate is better. The most useful comparison is between models tested on the same recordings, language, audio quality, and scoring method.
How should I compare speech-to-text APIs?
Compare APIs on the same recordings and scoring method, then consider word error rate, time to first text, final-text delay, streaming behavior, and failures together. Small differences from unrelated tests are not directly comparable.
Which datasets are used?
The benchmark reports the public Pipecat Dataset separately from eight licensed Ocular conversation recordings so differences between clean and conversational speech remain visible.
Can I compare two speech-to-text models directly?
Yes. The comparison tool provides head-to-head accuracy and streaming latency results for any two tested speech-to-text models.