CekuraBench
All comparisons
OpenAIOpenAI

GPT-Live 1 + GPT-6 Sol (low thinking) vs GPT Realtime 2.1 Mini

GPT-Live 1 + GPT-6 Sol (low thinking) is ahead on reliability, success rate, agent responses, data accuracy and response time, and GPT Realtime 2.1 Mini on stalled calls, interruption and cost.

Last updated Methodology by Dileep Chagam

Compare

Success rate by suite

Higher is better
Appointments59 scenariosMedicare23 scenarios
  • GPT-Live 1 + Sol lowOpenAI
    97%
    49%
  • Realtime 2.1 MiniOpenAI
    84%
    4%
Each model's share of calls that passed every check, on each suite.
Verdict

Which to pick

Pick GPT-Live 1 + GPT-6 Sol (low thinking) for reliability across repeated calls (25.6 points higher), calls that pass every check (22.4 points higher), getting the agent's replies right (6.5 points higher), saving the caller's data correctly (24.5 points higher) and fast replies (0.06s sooner). Pick GPT Realtime 2.1 Mini for fewer stalled calls (9.7 points lower), handling interruptions (0.10 higher) and lower cost ($0.035 cheaper).

Every metric, side by side

Reliability

Scenarios that passed on all three runs.

GPT-Live 1 + Sol low
75.6%2nd
Realtime 2.1 Mini
50.0%9th

GPT-Live 1 + Sol low, 25.6 points higher

Success rate

Calls that passed every check.

GPT-Live 1 + Sol low
83.7%3rd
Realtime 2.1 Mini
61.4%10th

GPT-Live 1 + Sol low, 22.4 points higher

Agent responses

Calls where the agent's replies reached the expected outcome.

GPT-Live 1 + Sol low
100.0%1st
Realtime 2.1 Mini
93.5%10th

GPT-Live 1 + Sol low, 6.5 points higher

Data accuracy

Calls where everything the agent saved matched exactly.

GPT-Live 1 + Sol low
89.0%2nd
Realtime 2.1 Mini
64.5%10th

GPT-Live 1 + Sol low, 24.5 points higher

Stalled calls

Calls where the agent went silent for 10 seconds or more.

GPT-Live 1 + Sol low
12.2%11th
Realtime 2.1 Mini
2.5%4th

Realtime 2.1 Mini, 9.7 points lower

Interruption

How well it handled being talked over, 0 to 5.

GPT-Live 1 + Sol low
4.8910th
Realtime 2.1 Mini
4.992nd

Realtime 2.1 Mini, 0.10 higher

Response time

Median time to start replying after the caller finishes.

GPT-Live 1 + Sol low
2.02s6th
Realtime 2.1 Mini
2.08s8th

GPT-Live 1 + Sol low, 0.06s sooner

Cost / min

One minute of call at the vendor's published rates.

GPT-Live 1 + Sol low
$0.0552nd
Realtime 2.1 Mini
$0.0201st

Realtime 2.1 Mini, $0.035 cheaper

The small figure is the place among ranked models. The cascade is not ranked.

GPT-Live 1 + Sol low answers first

GPT-Live 1 + Sol low 2.02s, Realtime 2.1 Mini 2.08s at the median.

Response time, median to p90

Lower is betterMedianTo p90
Medianp90
  • GPT-Live 1 + Sol lowOpenAI
    GPT-Live 1 + Sol low: median 2.02s, p90 2.42s
    2.02s2.42s
  • Realtime 2.1 MiniOpenAI
    Realtime 2.1 Mini: median 2.08s, p90 2.75s
    2.08s2.75s
2.0s3.0s
Seconds from the caller finishing to the agent starting to reply, per model.

What we saw on the calls

GPT-Live 1 + GPT-6 Sol (low thinking)

  • Near perfect on bookings, run after run.
All 4 notes on GPT-Live 1 + Sol low
Model
gpt-live-1
Setting
Delegates reasoning and tools to Sol
Voice
marin
Turn taking
Service turn detection
Audio
24 kHz
Cost
Includes the delegated backend model.

GPT Realtime 2.1 Mini

  • Gets most bookings right, at a fraction of the cost.
All 3 notes on Realtime 2.1 Mini
Model
gpt-realtime-2.1-mini
Setting
Reasoning high
Voice
marin
Turn taking
Service turn detection
Audio
24 kHz
Cost
Averaged over the calls with a usage record.
See the full leaderboardReference agent

Related comparisons

Frequently asked questions

Is GPT-Live 1 + GPT-6 Sol (low thinking) or GPT Realtime 2.1 Mini more reliable?

GPT-Live 1 + GPT-6 Sol (low thinking) is more reliable, by 25.6 points. GPT-Live 1 + GPT-6 Sol (low thinking) passed 62 of 82 scenarios on all three runs, and GPT Realtime 2.1 Mini passed 41. A scenario only counts when every run of it passed.

Which should I pick, GPT-Live 1 + GPT-6 Sol (low thinking) or GPT Realtime 2.1 Mini?

For a phone agent, GPT-Live 1 + GPT-6 Sol (low thinking), since it gets more scenarios right on every run. GPT-Live 1 + GPT-6 Sol (low thinking) is ahead on reliability, success rate, agent responses, data accuracy and response time, and GPT Realtime 2.1 Mini on stalled calls, interruption and cost.

Which saves caller data more accurately, GPT-Live 1 + GPT-6 Sol (low thinking) or GPT Realtime 2.1 Mini?

GPT-Live 1 + GPT-6 Sol (low thinking), by 24.5 points. GPT-Live 1 + GPT-6 Sol (low thinking) saved everything exactly right on 89.0% of the calls that save data, and GPT Realtime 2.1 Mini on 64.5%. On Appointments, 100.0% against 88.8%. On Medicare, 60.6% against 0.0%. A call counts only when nothing is missing, wrong, extra or saved twice.

Is GPT-Live 1 + GPT-6 Sol (low thinking) faster than GPT Realtime 2.1 Mini?

Yes. GPT-Live 1 + GPT-6 Sol (low thinking) starts replying 0.06s sooner at the median. GPT-Live 1 + GPT-6 Sol (low thinking) takes 2.02s at the median and 2.42s at p90; GPT Realtime 2.1 Mini takes 2.08s and 2.75s. Both are measured on the call audio, from the caller finishing to the agent starting to reply.

Which handles interruptions better, GPT-Live 1 + GPT-6 Sol (low thinking) or GPT Realtime 2.1 Mini?

GPT Realtime 2.1 Mini, by 0.10 on a 0 to 5 scale. GPT-Live 1 + GPT-6 Sol (low thinking) scores 4.89 and GPT Realtime 2.1 Mini 4.99 for how well the agent handled being talked over.

Which goes silent on callers less often, GPT-Live 1 + GPT-6 Sol (low thinking) or GPT Realtime 2.1 Mini?

GPT Realtime 2.1 Mini, by 9.7 points. The agent went silent for 10 seconds or more on 12.2% of GPT-Live 1 + GPT-6 Sol (low thinking) calls and 2.5% of GPT Realtime 2.1 Mini calls.

Which is cheaper, GPT-Live 1 + GPT-6 Sol (low thinking) or GPT Realtime 2.1 Mini?

GPT Realtime 2.1 Mini is $0.035 cheaper a minute. GPT-Live 1 + GPT-6 Sol (low thinking) costs $0.055 a minute of call and GPT Realtime 2.1 Mini $0.020, at each vendor's published rates.

Where does the GPT-Live 1 + GPT-6 Sol (low thinking) vs GPT Realtime 2.1 Mini data come from?

Cekura ran both as the whole agent in the same open-source Pipecat pipeline, with the same prompt, tools and connection. A simulated caller worked through all 82 scenarios on live calls, three times each, 2,952 calls across the benchmark. A call passes only when the agent said the right things and saved the right data.