AssemblyAI is ahead of OpenAI on both datasets together, the Pipecat Dataset, time to first text and final-text delay. The Ocular Dataset is too close to call.
Last updated Methodology by Dileep Chagam
Each row sets AssemblyAI's best model against OpenAI's best on that measure. For costs, see our breakdown of AssemblyAI pricing.
AssemblyAI is ahead, 0.48 points lower. The best AssemblyAI result is AssemblyAI STT 3.7 Realtime Agent at 1.68%, and the best OpenAI result is GPT Realtime Whisper at 2.16%, on the Pipecat Dataset.
AssemblyAI is ahead, 0.48 points lower. The best AssemblyAI result is AssemblyAI STT 3.7 Realtime Agent at 1.68%, and the best OpenAI result is GPT Realtime Whisper at 2.16%, on the Pipecat Dataset.
AssemblyAI is faster, 496ms sooner. The best AssemblyAI result is AssemblyAI STT 3.7 Realtime Agent at 47ms, and the best OpenAI result is GPT Realtime Whisper at 543ms, on final-text delay.
AssemblyAI: AssemblyAI STT 3.7 Realtime Agent, AssemblyAI Universal 3.5 Pro, AssemblyAI Universal 3.6 Pro. OpenAI: GPT-4o Mini Transcribe, GPT-4o Transcribe, GPT Realtime Whisper. Every model ran the same tests on Converse-STT.
Cekura runs these simulated calls on your agent and scores every one.