CekuraBench
Book a demo

Converse-STT benchmark

ResultsSource

Who wins what

The leader on each measure

  • Most accurateFish Audio Transcribe Stream 12.21%Overall WER
  • Real callsFish Audio Transcribe Stream 12.78%Ocular WER
  • Fastest final textSoniox STT-RT v538msTTFS P50
Full leaderboard

A caller stops talking. Soniox STT-RT v5 has the transcript ready in 38ms. Fish Audio Transcribe Stream 1 takes 123ms, but gets the fewest words wrong.

Last updated Methodology by Dileep Chagam

With Ocular

Leaderboard

#ModelWER
  1. Overall WER
    2.21%
    Both datasets
    Pipecat WER
    1.90%
    95% range 1.68 to 2.14%
    Ocular WER
    2.78%
    95% range 2.05 to 3.58%
    TTFT P50
    730ms
    P90 1.15s · P95 1.20s
    TTFS P50
    123ms
    P90 195ms · P95 253ms
    • $0.36 per hour of streamed audio. Fish Audio's published pay-as-you-go Realtime ASR rate of $0.006 per minute.
    Everything on Fish Audio Transcribe Stream 1 Compare with AssemblyAI Universal 3.6 Pro
  2. Overall WER
    2.28%
    Both datasets
    Pipecat WER
    1.89%
    95% range 1.63 to 2.17%
    Ocular WER
    3.03%
    95% range 2.26 to 3.79%
    TTFT P50
    n/a
    Not measured
    TTFS P50
    n/a
    Not measured
    • Open weights under OpenMDW-1.1, the only open-weights model on the board: anyone can download the model and run it on their own hardware.
    • Takes a whole recording per request rather than a live stream, so it has no TTFT or TTFS.
    • Every request used the verbatim transcription prompt from NVIDIA's model card, with reasoning off, and each 15-minute Ocular recording went in as a single request.
    • $0.12 per hour of audio. DeepInfra's hosted rate, $2 per 1,000 minutes. The scores come from a deployment hosted by NVIDIA, not DeepInfra's.
    Everything on NVIDIA Nemotron 3.5 Transcribe Compare with AssemblyAI Universal 3.6 Pro
  3. Overall WER
    2.38%
    Both datasets
    Pipecat WER
    2.10%
    95% range 1.86 to 2.35%
    Ocular WER
    2.93%
    95% range 2.29 to 3.68%
    TTFT P50
    2.40s
    P90 7.97s · P95 10.20s
    TTFS P50
    235ms
    P90 357ms · P95 398ms
    • $0.12 per hour of streamed audio. Reson8's real-time rate as of Oct 28, 2026.
    Everything on Reson8 Compare with AssemblyAI Universal 3.6 Pro
  4. Overall WER
    2.63%
    Both datasets
    Pipecat WER
    2.39%
    95% range 2.13 to 2.65%
    Ocular WER
    3.09%
    95% range 2.43 to 3.84%
    TTFT P50
    1.51s
    P90 2.04s · P95 3.23s
    TTFS P50
    123ms
    P90 139ms · P95 145ms
    • No published hourly price. No per-hour rate: Ink 2 is paid in subscription credits, 3 per second of audio.
    Everything on Cartesia Ink 2 Compare with AssemblyAI Universal 3.6 Pro
  5. Overall WER
    2.63%
    Both datasets
    Pipecat WER
    2.16%
    95% range 1.92 to 2.41%
    Ocular WER
    3.53%
    95% range 2.58 to 4.51%
    TTFT P50
    1.38s
    P90 1.84s · P95 2.86s
    TTFS P50
    543ms
    P90 647ms · P95 669ms
    • $1.02 per hour of streamed audio. OpenAI's estimated cost per minute for live transcription.
    Everything on GPT Realtime Whisper Compare with AssemblyAI Universal 3.6 Pro
  6. Overall WER
    2.71%
    Both datasets
    Pipecat WER
    1.77%
    95% range 1.55 to 2.00%
    Ocular WER
    4.50%
    95% range 3.66 to 5.34%
    TTFT P50
    485ms
    P90 966ms · P95 967ms
    TTFS P50
    91ms
    P90 210ms · P95 232ms
    • $0.45 per hour of streamed audio. Pay as you go, billed while the session is open.
    Everything on AssemblyAI Universal 3.6 Pro Compare with NVIDIA Nemotron 3.5 Transcribe
  7. Overall WER
    2.72%
    Both datasets
    Pipecat WER
    1.93%
    95% range 1.69 to 2.18%
    Ocular WER
    4.23%
    95% range 3.57 to 4.68%
    TTFT P50
    489ms
    P90 966ms · P95 1.08s
    TTFS P50
    180ms
    P90 532ms · P95 692ms
    • $0.45 per hour of streamed audio. AssemblyAI's standard real-time rate, billed while the session is open.
    Everything on AssemblyAI Universal 3.5 Pro Compare with AssemblyAI Universal 3.6 Pro
  8. Overall WER
    2.76%
    Both datasets
    Pipecat WER
    2.00%
    95% range 1.77 to 2.25%
    Ocular WER
    4.19%
    95% range 2.97 to 5.27%
    TTFT P50
    4.48s
    P90 6.21s · P95 7.97s
    TTFS P50
    475ms
    P90 941ms · P95 1.41s
    • TTFS is measured after the stream closes, because this service has no controlled finish.
    • $0.96 per hour of streamed audio. V2 Standard rate for the first 500,000 minutes a month; Chirp is billed as a Standard model.
    Everything on Google Chirp 3 Compare with AssemblyAI Universal 3.6 Pro
  9. Overall WER
    2.97%
    Both datasets
    Pipecat WER
    2.46%
    95% range 2.19 to 2.73%
    Ocular WER
    3.93%
    95% range 2.94 to 5.19%
    TTFT P50
    1.19s
    P90 1.56s · P95 2.52s
    TTFS P50
    147ms
    P90 192ms · P95 213ms
    • $0.30 per hour of streamed audio. Self-serve base rate, before the model-training and volume discounts.
    Everything on Speechmatics Linden Compare with AssemblyAI Universal 3.6 Pro
  10. Overall WER
    3.01%
    Both datasets
    Pipecat WER
    2.59%
    95% range 2.31 to 2.89%
    Ocular WER
    3.80%
    95% range 2.93 to 4.60%
    TTFT P50
    4.50s
    P90 6.40s · P95 8.15s
    TTFS P50
    568ms
    P90 1.04s · P95 1.22s
    • TTFS is measured after the stream closes, because this service has no controlled finish.
    • $0.96 per hour of streamed audio. V2 Standard rate for the first 500,000 minutes a month; Chirp is billed as a Standard model.
    Everything on Google Chirp 2 Compare with AssemblyAI Universal 3.6 Pro
  11. Overall WER
    3.02%
    Both datasets
    Pipecat WER
    2.54%
    95% range 2.27 to 2.83%
    Ocular WER
    3.92%
    95% range 2.60 to 4.86%
    TTFT P50
    1.20s
    P90 1.67s · P95 1.71s
    TTFS P50
    38ms
    P90 72ms · P95 79ms
    • $0.12 per hour of streamed audio. Real-time rate; billed per audio and text token.
    Everything on Soniox STT-RT v5 Compare with AssemblyAI Universal 3.6 Pro
  12. Overall WER
    3.59%
    Both datasets
    Pipecat WER
    3.66%
    95% range 3.34 to 3.99%
    Ocular WER
    3.46%
    95% range 2.66 to 4.22%
    TTFT P50
    1.06s
    P90 2.25s · P95 3.03s
    TTFS P50
    201ms
    P90 356ms · P95 773ms
    • $0.24 per hour of streamed audio. Pay as you go, published as about $0.004 a minute.
    Everything on Smallest Pulse Compare with AssemblyAI Universal 3.6 Pro
  13. Overall WER
    3.73%
    Both datasets
    Pipecat WER
    3.35%
    95% range 3.08 to 3.65%
    Ocular WER
    4.45%
    95% range 3.22 to 5.55%
    TTFT P50
    At speech end
    TTFS P50
    630ms
    P90 1.34s · P95 1.53s
    • Sends no text until the speaker stops, so its TTFT is not comparable.
    • $0.18 per hour of streamed audio. OpenAI's estimated cost per minute; billed per token.
    Everything on GPT-4o Mini Transcribe Compare with AssemblyAI Universal 3.6 Pro
  14. Overall WER
    3.78%
    Both datasets
    Pipecat WER
    3.46%
    95% range 3.16 to 3.78%
    Ocular WER
    4.39%
    95% range 3.42 to 5.25%
    TTFT P50
    1.06s
    P90 1.10s · P95 2.07s
    TTFS P50
    103ms
    P90 143ms · P95 155ms
    • $0.29 per hour of streamed audio. Pay as you go, current promotional rate; regular $0.462/hr.
    Everything on Deepgram Nova-3 Compare with AssemblyAI Universal 3.6 Pro
  15. Overall WER
    4.00%
    Both datasets
    Pipecat WER
    3.90%
    95% range 3.53 to 4.29%
    Ocular WER
    4.18%
    95% range 3.08 to 5.23%
    TTFT P50
    791ms
    P90 1.23s · P95 1.51s
    TTFS P50
    108ms
    P90 132ms · P95 141ms
    • $0.39 per hour of streamed audio. Pay as you go, current promotional rate; regular $0.462/hr.
    Everything on Deepgram Flux English Compare with AssemblyAI Universal 3.6 Pro
  16. Overall WER
    4.81%
    Both datasets
    Pipecat WER
    4.96%
    95% range 4.49 to 5.46%
    Ocular WER
    4.53%
    95% range 3.29 to 5.69%
    TTFT P50
    793ms
    P90 2.00s · P95 3.41s
    TTFS P50
    114ms
    P90 135ms · P95 141ms
    • $0.47 per hour of streamed audio. Pay as you go.
    Everything on Deepgram Flux Multilingual Compare with AssemblyAI Universal 3.6 Pro
  17. Overall WER
    5.44%
    Both datasets
    Pipecat WER
    2.72%
    95% range 1.67 to 4.68%
    Ocular WER
    10.60%
    95% range 3.85 to 22.76%
    TTFT P50
    1.06s
    P90 1.07s · P95 1.07s
    TTFS P50
    41ms
    P90 60ms · P95 65ms
    • $0.15 per hour of streamed audio. On-demand rate; $0.10/hr on paid plans.
    Everything on Inworld STT-1 Compare with AssemblyAI Universal 3.6 Pro
  18. Overall WER
    5.78%
    Both datasets
    Pipecat WER
    6.48%
    95% range 5.93 to 7.06%
    Ocular WER
    4.45%
    95% range 3.17 to 5.90%
    TTFT P50
    1.71s
    P90 4.05s · P95 7.32s
    TTFS P50
    212ms
    P90 266ms · P95 280ms
    • No published hourly price. No per-hour rate: speech-to-text is paid in plan credits, 3 per second of audio.
    Everything on Gradium Compare with AssemblyAI Universal 3.6 Pro
  19. Overall WER
    6.85%
    Both datasets
    Pipecat WER
    3.90%
    95% range 3.43 to 4.42%
    Ocular WER
    12.45%
    95% range 7.23 to 19.58%
    TTFT P50
    At speech end
    TTFS P50
    652ms
    P90 1.73s · P95 1.97s
    • Sends no text until the speaker stops, so its TTFT is not comparable.
    • $0.36 per hour of streamed audio. OpenAI's estimated cost per minute; billed per token.
    Everything on GPT-4o Transcribe Compare with AssemblyAI Universal 3.6 Pro

Lower is better on every measure. Overall WER counts every word in both datasets. Open a model for the Pipecat and Ocular WERs with their 95% ranges, and P90 and P95 timings. Prices are published rates, checked Oct 8, 2026.

Glossary: WER, TTFT, TTFS, P50
WER · Word error rate
Share of words the model got wrong: replaced, missing or added. Lower is better.
Pipecat Dataset
1,000 short public clips from Pipecat's STT benchmark dataset.
Ocular Dataset
8 recordings from 4 real phone conversations, licensed from Ocular.
Likely range · 95% range
Where the model's true score probably falls, given how much audio was tested. Overlapping ranges are too close to call.
TTFT · Time to first text
How long after the audio starts streaming until the first words appear.
TTFS · Final-text delay
How long after the caller stops until the full transcript arrives. A voice agent waits for this before it replies.
P50 / P90 / P95 · Typical, slow and slowest turns
Half of turns are faster than P50. Only 1 in 10 is slower than P90, and 1 in 20 slower than P95.
Price / hr · Published price
One hour of streamed audio at the vendor's published rate. ≈ marks a rate converted from another currency; a dash where no hourly rate is published.

Word error rate (WER)

Out of every 100 words a caller says, how many come out wrong. Each bar is one model; lower is better. Our guide explains how word error rate is measured.

  1. 2.2%
    Transcribe Stream 1
  2. 2.3%
    Nemotron 3.5 Transcribe
  3. 2.4%
    Reson8
  4. 2.6%
    Ink 2
  5. 2.6%
    Realtime Whisper
  6. 2.7%
    Universal 3.6 Pro
  7. 2.7%
    Universal 3.5 Pro
  8. 2.8%
    Chirp 3
  9. 3.0%
    Linden
  10. 3.0%
    Chirp 2
  11. 3.0%
    STT-RT v5
  12. 3.6%
    Pulse
  13. 3.7%
    4o Mini Transcribe
  14. 3.8%
    Nova-3
  15. 4.0%
    Flux English
  16. 4.8%
    Flux Multilingual
  17. 5.4%
    STT-1
  18. 5.8%
    Gradium
  19. 6.9%
    4o Transcribe

Pick by what your agent needs

Put two models head to head

Pick any two models and see them side by side. Every pair has its own page with the full numbers.

Compare
MeasureFish Audio Transcribe Stream 1Soniox STT-RT v5
  1. Overall WER2.21%3.02%
  2. Pipecat WER1.90%2.54%
  3. Ocular WER2.78%3.92%
  4. TTFT P50730ms1.20s
  5. TTFS P50123ms38ms
  6. Measures won4 of 51 of 5
Lower is better on every measure. The check marks the better value.All 171 comparisons

How we tested

Every model heard the same audio and was scored the same way.

  1. Step 1Same audioAll 19 models heard the same 1,000 public clips and the same real phone calls.
  2. Step 2Real calls tooRecordings licensed from Ocular show how models cope with real, messy conversation.
  3. Step 3Every word countsWord error rate counts every word replaced, missing or added against the reference transcript.
  4. Step 4Same clockSpeed is timed on the same streamed turns, typical case and slow case. Models that take only whole recordings are scored on accuracy alone.
Pipecat dataset Ocular Benchmark source

Speech-to-text benchmark FAQ

  • Fish Audio Transcribe Stream 1 has the lowest word error rate overall, at 2.21% across both datasets. AssemblyAI Universal 3.6 Pro has the lowest word error rate on the Pipecat Dataset, at 1.77%. 7 other models are within its margin of error there, so that lead is not certain. Fish Audio Transcribe Stream 1 has the lowest error rate on the Ocular conversation recordings, at 2.78%. 15 other models are within its margin of error there, so that lead is not certain. Soniox STT-RT v5 returns final text soonest, at 38ms median. Which is best depends on whether accuracy or speed matters more for your agent, so the leaderboard keeps the two apart.

  • It measures transcription accuracy with word error rate, streaming responsiveness with time to first text, and finalization speed with time to final text.

  • Lower word error rate is better. The most useful comparison is between models tested on the same recordings, language, audio quality, and scoring method.

  • Compare APIs on the same recordings and scoring method, then consider word error rate, time to first text, final-text delay, streaming behavior, and failures together. Small differences from unrelated tests are not directly comparable.

  • The benchmark reports the public Pipecat Dataset separately from eight licensed Ocular conversation recordings so differences between clean and conversational speech remain visible.

  • Yes. The comparison tool provides head-to-head accuracy and streaming latency results for any two tested speech-to-text models.

Test your voice agent the same way.

Cekura runs these simulated calls on your agent and scores every one.

Book a demoStart free