AssemblyAI vs OpenAI Whisper and GPT-4o Transcribe

AssemblyAI is ahead of OpenAI on both datasets together, the Pipecat Dataset, time to first text and final-text delay. The Ocular Dataset is too close to call.

Last updated Methodology by Dileep Chagam

Who is better at what

Each row sets AssemblyAI's best model against OpenAI's best on that measure. For costs, see our breakdown of AssemblyAI pricing.

AssemblyAIOpenAI
  • Overall WER2.41%AssemblyAI STT 3.7 Realtime Agent2.63%GPT Realtime Whisper
  • Pipecat Dataset WER1.68%AssemblyAI STT 3.7 Realtime Agent2.16%GPT Realtime Whisper
  • Ocular Dataset WER3.81%AssemblyAI STT 3.7 Realtime Agent3.53%GPT Realtime Whisper
  • Time to first text485msAssemblyAI Universal 3.6 Pro1.38sGPT Realtime Whisper
  • Final-text delay47msAssemblyAI STT 3.7 Realtime Agent543msGPT Realtime Whisper

AssemblyAI vs OpenAI FAQ

  • AssemblyAI is ahead, 0.48 points lower. The best AssemblyAI result is AssemblyAI STT 3.7 Realtime Agent at 1.68%, and the best OpenAI result is GPT Realtime Whisper at 2.16%, on the Pipecat Dataset.

  • AssemblyAI is faster, 496ms sooner. The best AssemblyAI result is AssemblyAI STT 3.7 Realtime Agent at 47ms, and the best OpenAI result is GPT Realtime Whisper at 543ms, on final-text delay.

  • AssemblyAI: AssemblyAI STT 3.7 Realtime Agent, AssemblyAI Universal 3.5 Pro, AssemblyAI Universal 3.6 Pro. OpenAI: GPT-4o Mini Transcribe, GPT-4o Transcribe, GPT Realtime Whisper. Every model ran the same tests on Converse-STT.

Test your voice agent the same way.

Cekura runs these simulated calls on your agent and scores every one.