CekuraBench

GPT Realtime 2.1 Mini

Every modelCompare

50.0%

Reliability

10th of 12

2.08s

Median reply

9th of 12

$0.020

Cost / min

1st of 9

Reliability vs. response time

Higher and further left is better

Best quadrant
Reliability, pass³ (%)
Median response time (s)

For the wider field, see our pick of the best voice AI APIs for real-time audio.

GPT Realtime 2.1 Mini against every other model

Scenarios that passed on all 3 runs, pass³. Higher is better.

#ModelReliability
  1. 01Grok Voice 3.081.7%
  2. 02GPT Realtime 2.179.3%
  3. 03GPT-Live 1 + GPT-6 Sol75.6%
  4. 04Azure Realtime70.7%
  5. 05Phonic v168.3%
  6. 06Grok Voice Think Fast 2.067.1%
  7. 07Gemini 3.1 Flash Live Preview62.2%
  8. 08Deepslate Opal57.3%
  9. 09Gemini 3.8 Live Extended Thinking52.4%
  10. 10Amazon Nova 2 Sonic50.0%
  11. 11GPT Realtime 2.1 Mini50.0%
  12. 12Gemini 3.8 Live41.5%
  • refCascade82.9%

Where GPT Realtime 2.1 Mini struggled

GPT Realtime 2.1 Mini falls furthest behind on tool or system errors: 25% passed, against a median of 92%.

Below the medianPassedMedian
  • Tool or system errors
    25%median 92%
  • Medicare compliance & routing
    0%median 40%
  • Plain booking & lookup
    89%median 94%
  • Safety & scope
    78%median 83%
  • Several requests in one call
    90%median 95%
  • Interruptions & barge-in
    87%median 90%
  • Caller corrects or changes mind
    79%median 81%
Mistakes in saved data
  • Field left out151 · 51%
  • Wrong value75 · 25%
  • Field that did not apply28 · 9%

Fields it got wrong most: route_id, zip_code, consent_id

Response time at P50, P90 and P95

GPT Realtime 2.1 Mini starts replying in 2.08s on a typical turn and 3.02s on the slowest turn in twenty, 9th fastest of 12 at the median.

ModelP50P90P95
  • Cascade
    2.17s2.51s2.62s
Seconds from the caller finishing to the agent replying. The bar is the typical turn, P50.

GPT Realtime 2.1 Mini head to head

Every comparisonAll 12 models

How much audio GPT Realtime 2.1 Mini was tested on

246 live calls: 82 scenarios, each run three times. The median call ran 2:03.

Live calls
246
Scenarios
82
Booking calls
177
Medicare calls
69
Replies per call
7.7
Median call
2:03

82 scenarios, the same for every model. See every test case

GPT Realtime 2.1 Mini FAQ

  • GPT Realtime 2.1 Mini passed 41 of 82 scenarios on all three runs, 50.0%, 10th of 12 realtime models. Across all 246 calls, 61.4% passed every check.

  • GPT Realtime 2.1 Mini starts replying 2.08s after the caller finishes at the median, 2.75s at P90 and 3.02s at P95. That is 9th fastest of 12 at the median, measured on the call audio.

  • GPT Realtime 2.1 Mini falls furthest behind the field on tool or system errors: 25% of calls passed, against a median of 92%. When it saved data wrong, the most common mistake was field left out, 51% of them.

  • GPT Realtime 2.1 Mini saved everything exactly right on 64.5% of the calls that save data, 11th of 12. On Booking calls, 88.8%. On Medicare calls, 0.0%.

  • $0.020 a minute of call at OpenAI's published rates, 1st cheapest of 9 priced models. Averaged over the calls with a usage record.

  • Cekura ran GPT Realtime 2.1 Mini as the whole agent in the same open-source Pipecat pipeline as every other model, with the same prompt, tools and connection. A simulated caller worked through 82 scenarios on live calls, three times each: 246 calls for GPT Realtime 2.1 Mini. A call passes only when the agent said the right things and saved the right data.