CekuraBench
All comparisons
OpenAIPhonic

GPT Realtime 2.1 Mini vs Phonic v1

Phonic v1 is ahead on reliability, success rate, agent responses, data accuracy, stalled calls and response time, and GPT Realtime 2.1 Mini on interruption and cost.

Last updated Methodology by Dileep Chagam

Compare

Success rate by suite

Higher is better
Appointments59 scenariosMedicare23 scenarios
  • Phonic v1Phonic
    97%
    26%
  • Realtime 2.1 MiniOpenAI
    84%
    4%
Each model's share of calls that passed every check, on each suite.
Verdict

Which to pick

Pick GPT Realtime 2.1 Mini for handling interruptions (0.10 higher) and lower cost ($0.120 cheaper). Pick Phonic v1 for reliability across repeated calls (18.3 points higher), calls that pass every check (15.9 points higher), getting the agent's replies right (4.5 points higher), saving the caller's data correctly (12.9 points higher), fewer stalled calls (2.5 points lower) and fast replies (0.48s sooner).

Every metric, side by side

Reliability

Scenarios that passed on all three runs.

Realtime 2.1 Mini
50.0%9th
Phonic v1
68.3%4th

Phonic v1, 18.3 points higher

Success rate

Calls that passed every check.

Realtime 2.1 Mini
61.4%10th
Phonic v1
77.2%4th

Phonic v1, 15.9 points higher

Agent responses

Calls where the agent's replies reached the expected outcome.

Realtime 2.1 Mini
93.5%10th
Phonic v1
98.0%4th

Phonic v1, 4.5 points higher

Data accuracy

Calls where everything the agent saved matched exactly.

Realtime 2.1 Mini
64.5%10th
Phonic v1
77.4%7th

Phonic v1, 12.9 points higher

Stalled calls

Calls where the agent went silent for 10 seconds or more.

Realtime 2.1 Mini
2.5%4th
Phonic v1
0.0%1st

Phonic v1, 2.5 points lower

Interruption

How well it handled being talked over, 0 to 5.

Realtime 2.1 Mini
4.992nd
Phonic v1
4.8911th

Realtime 2.1 Mini, 0.10 higher

Response time

Median time to start replying after the caller finishes.

Realtime 2.1 Mini
2.08s8th
Phonic v1
1.60s1st

Phonic v1, 0.48s sooner

Cost / min

One minute of call at the vendor's published rates.

Realtime 2.1 Mini
$0.0201st
Phonic v1
$0.1408th

Realtime 2.1 Mini, $0.120 cheaper

The small figure is the place among ranked models. The cascade is not ranked.

Phonic v1 answers first

Phonic v1 1.60s, Realtime 2.1 Mini 2.08s at the median.

Response time, median to p90

Lower is betterMedianTo p90
Medianp90
  • Phonic v1Phonic
    Phonic v1: median 1.60s, p90 1.88s
    1.60s1.88s
  • Realtime 2.1 MiniOpenAI
    Realtime 2.1 Mini: median 2.08s, p90 2.75s
    2.08s2.75s
2.0s3.0s
Seconds from the caller finishing to the agent starting to reply, per model.

What we saw on the calls

GPT Realtime 2.1 Mini

  • Gets most bookings right, at a fraction of the cost.
All 3 notes on Realtime 2.1 Mini
Model
gpt-realtime-2.1-mini
Setting
Reasoning high
Voice
marin
Turn taking
Service turn detection
Audio
24 kHz
Cost
Averaged over the calls with a usage record.

Phonic v1

  • Quickest to reply, and never left a caller waiting.
All 4 notes on Phonic v1
Model
phonic_v1
Setting
Intelligence high
Voice
sabrina
Turn taking
Service turn detection
Audio
16 kHz
Cost
The vendor's Starter plan rate per conversation minute, applied to each call's length; the Pro plan is $0.12.
See the full leaderboardReference agent

Related comparisons

Frequently asked questions

Is GPT Realtime 2.1 Mini or Phonic v1 more reliable?

Phonic v1 is more reliable, by 18.3 points. GPT Realtime 2.1 Mini passed 41 of 82 scenarios on all three runs, and Phonic v1 passed 56. A scenario only counts when every run of it passed.

Which should I pick, GPT Realtime 2.1 Mini or Phonic v1?

For a phone agent, Phonic v1, since it gets more scenarios right on every run. Phonic v1 is ahead on reliability, success rate, agent responses, data accuracy, stalled calls and response time, and GPT Realtime 2.1 Mini on interruption and cost.

Which saves caller data more accurately, GPT Realtime 2.1 Mini or Phonic v1?

Phonic v1, by 12.9 points. GPT Realtime 2.1 Mini saved everything exactly right on 64.5% of the calls that save data, and Phonic v1 on 77.4%. On Appointments, 88.8% against 98.2%. On Medicare, 0.0% against 25.0%. A call counts only when nothing is missing, wrong, extra or saved twice.

Is GPT Realtime 2.1 Mini faster than Phonic v1?

No. Phonic v1 starts replying 0.48s sooner at the median. GPT Realtime 2.1 Mini takes 2.08s at the median and 2.75s at p90; Phonic v1 takes 1.60s and 1.88s. Both are measured on the call audio, from the caller finishing to the agent starting to reply.

Which handles interruptions better, GPT Realtime 2.1 Mini or Phonic v1?

GPT Realtime 2.1 Mini, by 0.10 on a 0 to 5 scale. GPT Realtime 2.1 Mini scores 4.99 and Phonic v1 4.89 for how well the agent handled being talked over.

Which goes silent on callers less often, GPT Realtime 2.1 Mini or Phonic v1?

Phonic v1, by 2.5 points. The agent went silent for 10 seconds or more on 2.5% of GPT Realtime 2.1 Mini calls and 0.0% of Phonic v1 calls.

Which is cheaper, GPT Realtime 2.1 Mini or Phonic v1?

GPT Realtime 2.1 Mini is $0.120 cheaper a minute. GPT Realtime 2.1 Mini costs $0.020 a minute of call and Phonic v1 $0.140, at each vendor's published rates.

Where does the GPT Realtime 2.1 Mini vs Phonic v1 data come from?

Cekura ran both as the whole agent in the same open-source Pipecat pipeline, with the same prompt, tools and connection. A simulated caller worked through all 82 scenarios on live calls, three times each, 2,952 calls across the benchmark. A call passes only when the agent said the right things and saved the right data.