CekuraBench
All comparisons
OpenAICascade baseline

GPT Realtime 2.1 Mini vs Flux → GPT-4.1 → ElevenLabs Flash

Flux → GPT-4.1 → ElevenLabs Flash is ahead on reliability, success rate, agent responses, data accuracy and stalled calls, and GPT Realtime 2.1 Mini on interruption and response time.

Last updated Methodology by Dileep Chagam

Compare

Success rate by suite

Higher is better
Appointments59 scenariosMedicare23 scenarios
  • Realtime 2.1 MiniOpenAI
    84%
    4%
  • Cascade, GPT-4.1Not ranked
    97%
    83%
Each model's share of calls that passed every check, on each suite.
Verdict

Which to pick

Pick GPT Realtime 2.1 Mini for handling interruptions (0.03 higher) and fast replies (0.09s sooner). Pick Flux → GPT-4.1 → ElevenLabs Flash for reliability across repeated calls (32.9 points higher), calls that pass every check (31.7 points higher), getting the agent's replies right (5.7 points higher), saving the caller's data correctly (29.6 points higher) and fewer stalled calls (1.6 points lower).

Every metric, side by side

Reliability

Scenarios that passed on all three runs.

Realtime 2.1 Mini
50.0%9th
Cascade, GPT-4.1
82.9%

Cascade, GPT-4.1, 32.9 points higher

Success rate

Calls that passed every check.

Realtime 2.1 Mini
61.4%10th
Cascade, GPT-4.1
93.1%

Cascade, GPT-4.1, 31.7 points higher

Agent responses

Calls where the agent's replies reached the expected outcome.

Realtime 2.1 Mini
93.5%10th
Cascade, GPT-4.1
99.2%

Cascade, GPT-4.1, 5.7 points higher

Data accuracy

Calls where everything the agent saved matched exactly.

Realtime 2.1 Mini
64.5%10th
Cascade, GPT-4.1
94.1%

Cascade, GPT-4.1, 29.6 points higher

Stalled calls

Calls where the agent went silent for 10 seconds or more.

Realtime 2.1 Mini
2.5%4th
Cascade, GPT-4.1
0.8%

Cascade, GPT-4.1, 1.6 points lower

Interruption

How well it handled being talked over, 0 to 5.

Realtime 2.1 Mini
4.992nd
Cascade, GPT-4.1
4.96

Realtime 2.1 Mini, 0.03 higher

Response time

Median time to start replying after the caller finishes.

Realtime 2.1 Mini
2.08s8th
Cascade, GPT-4.1
2.17s

Realtime 2.1 Mini, 0.09s sooner

Cost / min

One minute of call at the vendor's published rates.

Realtime 2.1 Mini
$0.0201st
Cascade, GPT-4.1
No verified rate

Not comparable

The small figure is the place among ranked models. The cascade is not ranked.

Realtime 2.1 Mini answers first

Realtime 2.1 Mini 2.08s, Cascade, GPT-4.1 2.17s at the median.

Response time, median to p90

Lower is betterMedianTo p90
Medianp90
  • Realtime 2.1 MiniOpenAI
    Realtime 2.1 Mini: median 2.08s, p90 2.75s
    2.08s2.75s
  • Cascade, GPT-4.1Not ranked
    Cascade, GPT-4.1: median 2.17s, p90 2.51s
    2.17s2.51s
2.0s3.0s
Seconds from the caller finishing to the agent starting to reply, per model.

What we saw on the calls

GPT Realtime 2.1 Mini

  • Gets most bookings right, at a fraction of the cost.
All 3 notes on Realtime 2.1 Mini
Model
gpt-realtime-2.1-mini
Setting
Reasoning high
Voice
marin
Turn taking
Service turn detection
Audio
24 kHz
Cost
Averaged over the calls with a usage record.

Flux → GPT-4.1 → ElevenLabs Flash

  • Most scenarios pass every run, and none fails every run.
All 4 notes on Cascade, GPT-4.1
Model
flux-general-en → gpt-4.1 → eleven_flash_v2_5
Setting
No reasoning setting
Voice
ElevenLabs 21m00Tcm4TlvDq8ikWAM
Turn taking
Speech-to-text turn detection
Audio
16 kHz
Cost
Not published: no verified combined rate for the three services.
See the full leaderboardReference agent

Related comparisons

Frequently asked questions

Is GPT Realtime 2.1 Mini or Flux → GPT-4.1 → ElevenLabs Flash more reliable?

Flux → GPT-4.1 → ElevenLabs Flash is more reliable, by 32.9 points. GPT Realtime 2.1 Mini passed 41 of 82 scenarios on all three runs, and Flux → GPT-4.1 → ElevenLabs Flash passed 68. A scenario only counts when every run of it passed.

Is GPT Realtime 2.1 Mini better than a cascaded STT, LLM and TTS pipeline?

On reliability, no: the Flux → GPT-4.1 → ElevenLabs Flash cascade passes more scenarios on every run. Flux → GPT-4.1 → ElevenLabs Flash is ahead on reliability, success rate, agent responses, data accuracy and stalled calls, and GPT Realtime 2.1 Mini on interruption and response time. Both ran as the same agent, with the same prompt and tools.

Which saves caller data more accurately, GPT Realtime 2.1 Mini or Flux → GPT-4.1 → ElevenLabs Flash?

Flux → GPT-4.1 → ElevenLabs Flash, by 29.6 points. GPT Realtime 2.1 Mini saved everything exactly right on 64.5% of the calls that save data, and Flux → GPT-4.1 → ElevenLabs Flash on 94.1%. On Appointments, 88.8% against 98.8%. On Medicare, 0.0% against 81.8%. A call counts only when nothing is missing, wrong, extra or saved twice.

Is GPT Realtime 2.1 Mini faster than Flux → GPT-4.1 → ElevenLabs Flash?

Yes. GPT Realtime 2.1 Mini starts replying 0.09s sooner at the median. GPT Realtime 2.1 Mini takes 2.08s at the median and 2.75s at p90; Flux → GPT-4.1 → ElevenLabs Flash takes 2.17s and 2.51s. Both are measured on the call audio, from the caller finishing to the agent starting to reply.

Which handles interruptions better, GPT Realtime 2.1 Mini or Flux → GPT-4.1 → ElevenLabs Flash?

GPT Realtime 2.1 Mini, by 0.03 on a 0 to 5 scale. GPT Realtime 2.1 Mini scores 4.99 and Flux → GPT-4.1 → ElevenLabs Flash 4.96 for how well the agent handled being talked over.

Which goes silent on callers less often, GPT Realtime 2.1 Mini or Flux → GPT-4.1 → ElevenLabs Flash?

Flux → GPT-4.1 → ElevenLabs Flash, by 1.6 points. The agent went silent for 10 seconds or more on 2.5% of GPT Realtime 2.1 Mini calls and 0.8% of Flux → GPT-4.1 → ElevenLabs Flash calls.

Which is cheaper, GPT Realtime 2.1 Mini or Flux → GPT-4.1 → ElevenLabs Flash?

There is no verified per-minute rate for Flux → GPT-4.1 → ElevenLabs Flash, so cost cannot be compared yet.

Where does the GPT Realtime 2.1 Mini vs Flux → GPT-4.1 → ElevenLabs Flash data come from?

Cekura ran both as the whole agent in the same open-source Pipecat pipeline, with the same prompt, tools and connection. A simulated caller worked through all 82 scenarios on live calls, three times each, 2,952 calls across the benchmark. A call passes only when the agent said the right things and saved the right data. The cascade runs the same agent for reference and is never ranked.