The open-source model was more reliable and faster
Kimi K2.6 reached 88.1% pass³ against GPT-4.1’s 76.3%. It also cut median turn latency from 2.46s to 1.44s and P95 latency from 5.00s to 3.22s. Voice tone scores were nearly identical, and both variants shared the same TTS voice.
Complete head-to-head results
| Variant | LLM | pass¹ | pass³ | P50 | P95 | Interrupt | End call | Voice tone | Report |
|---|---|---|---|---|---|---|---|---|---|
| Telnyx · gpt-4.1 | gpt-4.1 (archived v0 configuration) | 89.8% | 76.3% | 2.46s | 5.00s | 4.74 | 96.6% | 4.47 | View runs ↗ |
| Telnyx · Kimi K2.6 | Kimi K2.6 1T A32B (self-hosted OSS) | 94.4% | 88.1% | 1.44s | 3.22s | 4.82 | 98.9% | 4.46 | View runs ↗ |
pass³ by evaluator category
| Category | gpt-4.1 | Kimi K2.6 |
|---|---|---|
| Positive / Core Scheduling | 50.0% | 75.0% |
| Workflow Complexity & Recovery | 85.7% | 90.5% |
| Voice Robustness & Turn-Taking | 72.0% | 92.0% |
| Red Team, Safety & Privacy | 100.0% | 80.0% |
Only the LLM changed
The Telnyx platform, SHA-verified prompt, first message, four tool definitions, mock data, voice, STT, TTS, and 59-evaluator suite remained fixed. Each model ran 177 calls: 59 evaluators repeated three times. This controlled v0-era experiment remains separate from the current seven-configuration leaderboard because its cohort and methodology differ.
Frequently asked questions
Did Kimi K2.6 outperform GPT-4.1 in this voice-agent test?+
Yes. With the platform, prompt, tools, voice, STT, and TTS held constant, Kimi K2.6 reached 88.1% pass³ versus GPT-4.1 at 76.3%, with 1.44s versus 2.46s median turn latency.
What changed between the two Telnyx variants?+
Only the LLM. The system prompt, first message, four tool definitions, mock data, voice, speech-to-text, text-to-speech, evaluators, and Telnyx orchestration remained the same.
Where did GPT-4.1 perform better?+
GPT-4.1 retained the advantage on Red Team, Safety & Privacy, scoring 100.0% pass³ versus Kimi K2.6 at 80.0%.