921
ELOELO Rating
10th of 12
14.3%
Pass³Reliability (Pass³)
10th of 12
2.40s
LatencyLatency P50
9th of 12
$0.020
Cost/MinCost per Minute (USD)
1st of 8
Higher wins more head-to-head calls (1000 = average); further left replies faster
Each model took the same recorded human callers on live calls. Pick a measure to see where GPT Realtime 2.1 Mini ranks.
Rating from head-to-head calls: on every call each pair of models is compared, the better call wins. 1000 is the average model. Higher is better.
Pick a skill family. The shape is this model on every skill in it; the dashed line is the median of the other models.
Share of calls where the skill was needed and handled, strongest first. Hover a point on the chart for both values.
Banking and healthcare callers. Each shows what the caller needed, what made the call hard, and how the model handled it.
Finished 3 of 6 steps
Handled well: Emotion Recognition, Overlapping-Speech Recovery, Tool-Latency Handling
Struggled with: Multi-Speaker Handling
Listen to the callFinished 5 of 7 steps
Handled well: Side-Speech Rejection, Mid-Utterance Pause Tolerance, Redundant Re-prompt Avoidance
Struggled with: Tool-Latency Handling, Consent Verification
Listen to the callFinished 5 of 7 steps
Handled well: Multi-Speaker Handling, Overlapping-Speech Recovery, Correction Handling
Struggled with: Degraded-Channel Robustness
Listen to the callFinished 5 of 7 steps
Handled well: Degraded-Channel Robustness, Backchannel vs Barge-in Detection, Consent Verification
Listen to the callFinished 2 of 7 steps
Handled well: Prosodic Cue Understanding, Mid-Utterance Pause Tolerance, Redundant Re-prompt Avoidance
Struggled with: Low-Confidence Confirmation, Consent Verification, Cancellation Handling
Listen to the callFinished 3 of 7 steps
Handled well: Waiting Through Hold, Multi-Speaker Handling, Redundant Re-prompt Avoidance
Struggled with: Human Handoff, Consent Verification
Listen to the callFinished 2 of 3 steps
Handled well: Tool-Latency Handling, Mid-Utterance Pause Tolerance, Redundant Re-prompt Avoidance
Struggled with: Premature-Response Avoidance
Listen to the callFinished 4 of 6 steps
Handled well: Cancellation Handling, Redundant Re-prompt Avoidance, Hallucinated-Success Avoidance
Struggled with: Tool-Latency Handling, Consent Verification
Listen to the callFinished 3 of 6 steps
Handled well: Prosodic Cue Understanding, Consent Verification, Redundant Re-prompt Avoidance
Struggled with: Multi-Speaker Handling, Endpointing, Premature-Response Avoidance
Listen to the callFinished 6 of 8 steps
Handled well: Side-Speech Rejection, Backchannel vs Barge-in Detection, Correction Handling
Struggled with: Low-Confidence Confirmation
Listen to the callGPT Realtime 2.1 Mini starts replying in 2.40s on a typical turn and 3.00s on the slowest turn in twenty, 9th fastest of 12 at the median.
210 live calls: 70 scenarios, each run three times. The median call ran 5:39.
70 scenarios, the same for every model.
GPT Realtime 2.1 Mini has an ELO rating of 921, 10th of 12 ranked models. It passed 10 of 70 callers on all three runs, a Pass³ of 14.3%.
GPT Realtime 2.1 Mini has an ELO rating of 921, 10th of 12 ranked models. It passed 10 of 70 callers on all three runs, a Pass³ of 14.3%.
GPT Realtime 2.1 Mini starts replying 2.40s after the caller finishes at P50, 2.88s at P90 and 3.00s at P95: 9th fastest of 12 at P50, measured on the call audio.
GPT Realtime 2.1 Mini falls furthest behind on Entity-Heavy: 46%, against a median of 61%.
GPT Realtime 2.1 Mini has a task completion rate of 13.8%, 10th of 12. A call counts only when every step the caller needed was done.
$0.020 per minute of call at OpenAI's published rates, 1st cheapest of 8 priced models. Averaged over the calls with a usage record.
Cekura ran GPT Realtime 2.1 Mini as the whole agent with the same prompt, tools and phone connection as every other model. It took the same 70 recorded real human callers on live calls, three times each (210 calls), and each call was graded on 25 skills across Hearing, Turn Taking, Task Handling and Speaking.
Cekura runs these simulated calls on your agent and scores every one.
