Google Chirp 2vsGoogle Chirp 3
Accuracy and speed
Left gets fewer words wrong. Lower has the final text ready sooner.
- Chirp 22.59% · 568ms
- Chirp 32.00% · 475ms
Google Chirp 3 is ahead of Google Chirp 2 on the Pipecat Dataset, time to first text and final-text delay. The Ocular Dataset is too close to call.
Last updated Methodology by Dileep Chagam
Who is better at what
- Pipecat Dataset WER2.59%2.00%
- Ocular Dataset WER3.80%4.19%
- Time to first text4.50s4.48s
- Final-text delay568ms475ms
Lower is better on every measure. Too close to call means the likely ranges overlap.
Chirp 2 vs Chirp 3 FAQ
Which is more accurate, Google Chirp 2 or Google Chirp 3?
Google Chirp 3 is more accurate on the public clips. On the 1,000 public Pipecat clips, Google Chirp 2 scores 2.59% and Google Chirp 3 2.00%, so Google Chirp 3 makes fewer errors by 0.59 points. On the eight licensed Ocular recordings, Google Chirp 2 scores 3.80% and Google Chirp 3 4.19%, with likely ranges of 2.93 to 4.60% and 2.97 to 5.27%. The ranges overlap, so the difference is too close to call. Word error rate counts incorrect, missing, and extra words, so lower is better.
Google Chirp 3 is more accurate on the public clips. On the 1,000 public Pipecat clips, Google Chirp 2 scores 2.59% and Google Chirp 3 2.00%, so Google Chirp 3 makes fewer errors by 0.59 points. On the eight licensed Ocular recordings, Google Chirp 2 scores 3.80% and Google Chirp 3 4.19%, with likely ranges of 2.93 to 4.60% and 2.97 to 5.27%. The ranges overlap, so the difference is too close to call. Word error rate counts incorrect, missing, and extra words, so lower is better.
No. Google Chirp 3 finishes the transcript 93ms sooner. Google Chirp 2 has the final transcript 568ms after the speaker stops and Google Chirp 3 475ms, at the median. Both are measured on the same 206 streamed turns.
Google Chirp 3 finishes the transcript 93ms sooner, and the two are too close to call on real conversations. A voice agent cannot reply until the final transcript arrives, and the Ocular recordings are real conversations.
Yes, both return partial text before the speaker stops. Google Chirp 2 shows first text after 4.50s and Google Chirp 3 after 4.48s, at the median, so Google Chirp 3 is 17ms sooner.