GoogleOpenAI

Gemini 3.8 Live vs GPT-Live 1 + GPT-6 Sol (low thinking)

GPT-Live 1 + GPT-6 Sol (low thinking) is ahead on reliability, success rate, agent responses and data accuracy, and Gemini 3.8 Live on stalled calls, interruption and response time.

Compare

Success rate by suite

Appointments59 scenariosMedicare23 scenarios

GPT Realtime 2.1OpenAI
92%
81%
GPT-Live 1 + Sol lowOpenAI
97%
49%
Phonic v1Phonic
97%
26%
Gemini 3.8 Live ThinkingGoogle
79%
70%
Grok Think Fast 2.0xAI
96%
25%
Gemini 3.1 Flash LiveGoogle
93%
32%
Gemini 3.8 LiveGoogle
76%
52%
Realtime 2.1 MiniOpenAI
84%
4%
Nova 2 SonicAmazon
80%
6%

Cascade, GPT-4.1Not ranked
97%
83%

Each model's share of calls that passed every check, on each suite.

Every metric, side by side

Reliability

Scenarios that passed on all three runs.

Gemini 3.8 Live: 41.5%9th
GPT-Live 1 + Sol low: 75.6%2nd

GPT-Live 1 + Sol low, 34.1 points higher

Success rate

Calls that passed every check.

Gemini 3.8 Live: 69.1%7th
GPT-Live 1 + Sol low: 83.7%2nd

GPT-Live 1 + Sol low, 14.6 points higher

Agent responses

Calls where the agent's replies reached the expected outcome.

Gemini 3.8 Live: 94.3%6th
GPT-Live 1 + Sol low: 100.0%1st

GPT-Live 1 + Sol low, 5.7 points higher

Data accuracy

Calls where everything the agent saved matched exactly.

Gemini 3.8 Live: 70.5%7th
GPT-Live 1 + Sol low: 89.0%2nd

GPT-Live 1 + Sol low, 18.6 points higher

Stalled calls

Calls where the agent went silent for 10 seconds or more.

Gemini 3.8 Live: 2.0%3rd
GPT-Live 1 + Sol low: 12.2%9th

Gemini 3.8 Live, 10.2 points lower

Interruption

How well it handled being talked over, 0 to 5.

Gemini 3.8 Live: 4.983rd
GPT-Live 1 + Sol low: 4.898th

Gemini 3.8 Live, 0.09 higher

Response time

Median time to start replying after the caller finishes.

Gemini 3.8 Live: 1.91s3rd
GPT-Live 1 + Sol low: 2.02s5th

Gemini 3.8 Live, 0.11s sooner

Cost / min

One minute of call at the vendor's published rates.

Gemini 3.8 Live: No verified rate
GPT-Live 1 + Sol low: $0.0552nd

Not comparable

The small figure is the place among ranked models. The cascade is not ranked.

Gemini 3.8 Live answers first

Gemini 3.8 Live 1.91s, GPT-Live 1 + Sol low 2.02s at the median.

Response time, median to p90

MedianTo p90

Medianp90

Phonic v1Phonic
Phonic v1: median 1.60s, p90 1.88s
1.60s1.88s
Grok Think Fast 2.0xAI
Grok Think Fast 2.0: median 1.62s, p90 1.94s
1.62s1.94s
Gemini 3.8 LiveGoogle
Gemini 3.8 Live: median 1.91s, p90 2.23s
1.91s2.23s
GPT Realtime 2.1OpenAI
GPT Realtime 2.1: median 1.94s, p90 2.46s
1.94s2.46s
GPT-Live 1 + Sol lowOpenAI
GPT-Live 1 + Sol low: median 2.02s, p90 2.42s
2.02s2.42s
Gemini 3.8 Live ThinkingGoogle
Gemini 3.8 Live Thinking: median 2.07s, p90 2.74s
2.07s2.74s
Realtime 2.1 MiniOpenAI
Realtime 2.1 Mini: median 2.08s, p90 2.75s
2.08s2.75s
Nova 2 SonicAmazon
Nova 2 Sonic: median 2.18s, p90 2.76s
2.18s2.76s
Gemini 3.1 Flash LiveGoogle
Gemini 3.1 Flash Live: median 2.88s, p90 4.00s
2.88s4.00s
Cascade, GPT-4.1Not ranked
Cascade, GPT-4.1: median 2.17s, p90 2.51s
2.17s2.51s

1.5s2.0s2.5s3.0s3.5s4.0s

Seconds from the caller finishing to the agent starting to reply, per model.

What we saw on the calls

Gemini 3.8 Live

Without extended thinking it replies sooner, and short-form calls reach the outcome more often.
On the long form the conversation still reaches the outcome, but the saved record is wrong far more often.
Weakest of the field when the caller corrects themselves.
Google's turn detection stalls it under background conversation: the caller finishes and no reply comes (AS50, AS52, AS14).
Least predictable of the field.

Model: gemini-3.8-live
Setting: No thinking level
Voice: Charon
Turn taking: Service turn detection
Audio: 16 kHz
Cost: The export carries audio tokens but not text or cached input, so a per-call cost is not derived.

GPT-Live 1 + GPT-6 Sol (low thinking)

Near perfect on the short form, run after run.
Every failure is in the saved record; the conversation itself always reaches the outcome.
On the long form it routes the caller wrong or skips a step.
Says goodbye without hanging up, and stalls more than any other model.
Lowest voice score of the field.

Model: gpt-live-1
Setting: Delegates reasoning and tools to Sol
Voice: marin
Turn taking: Service turn detection
Audio: 24 kHz
Cost: Includes the delegated backend model.

See the full leaderboard Reference agent

Related comparisons

Frequently asked questions

Is Gemini 3.8 Live or GPT-Live 1 + GPT-6 Sol (low thinking) more reliable?

Gemini 3.8 Live passed 34 of 82 scenarios on all three runs, and GPT-Live 1 + GPT-6 Sol (low thinking) passed 62. GPT-Live 1 + GPT-6 Sol (low thinking) is ahead, by 34.1 points.

Which saves caller data more accurately, Gemini 3.8 Live or GPT-Live 1 + GPT-6 Sol (low thinking)?

Gemini 3.8 Live saved everything exactly right on 70.5% of the calls that save data, and GPT-Live 1 + GPT-6 Sol (low thinking) on 89.0%. On Appointments, 78.4% against 100.0%. On Medicare, 50.0% against 60.6%. A call counts only when nothing is missing, wrong, extra or saved twice.

Which responds faster, Gemini 3.8 Live or GPT-Live 1 + GPT-6 Sol (low thinking)?

Gemini 3.8 Live starts replying 0.11s sooner at the median. Gemini 3.8 Live takes 1.91s at the median and 2.23s at p90; GPT-Live 1 + GPT-6 Sol (low thinking) takes 2.02s and 2.42s. Both are measured on the call audio, from the caller finishing to the agent starting to reply.

Which costs less, Gemini 3.8 Live or GPT-Live 1 + GPT-6 Sol (low thinking)?

There is no verified per-minute rate for Gemini 3.8 Live, so cost cannot be compared yet.

How were Gemini 3.8 Live and GPT-Live 1 + GPT-6 Sol (low thinking) compared?

Both ran as the whole agent in the same open-source Pipecat pipeline, with the same prompt, tools and connection. A simulated caller worked through all 82 scenarios on live calls, three times each, 2,460 calls across the benchmark. A call passes only when the agent said the right things and saved the right data.