Google Assistant tops Siri and Alexa in ‘Smart Speaker IQ’ test, but by a narrowing margin

Ben Schoon | Dec 20 2018 - 9:20 am PT

It’s hard for anyone to argue against Google Assistant being the best voice assistant at answering queries. Assistant has proven time after time that it’s superior. Recently though, one test found that the gap between Assistant, Siri, and Alexa is starting to close quite a bit.

Loup Ventures has, for the second year in a row, performed a test with Google Assistant, Siri, and Amazon Alexa smart speakers to determine which is the smartest option. Unsurprisingly, Google Assistant took the crown here yet again. Over 800 questions, Google Assistant understood 100% of what it was asked, answering correctly just shy of 88% of the time.

Apple’s Siri, on the other hand, answered just 74.6% of the questions correctly and understood 99.6%. Alexa, finally, understood 99% of the questions asked, answering 72.5% of them correctly. In a distant fourth place, Microsoft’s Cortana understood 99.4% of the questions but answered only 63.4% of them correctly.

Google Home continued its outperformance, answering 86% correctly and understanding all 800 questions. The HomePod correctly answered 75% and only misunderstood 3, the Echo correctly answered 73% and misunderstood 8 questions, and Cortana correctly answered 63% and misunderstood just 5 questions.

These numbers shouldn’t be all that surprising, and they actually align fairly closely to what we’ve seen from other, similar tests. Notably, Loup Ventures used an Amazon Echo, Google Home Mini, Apple HomePod, and Harmon Kardon Invoke to perform this test. The questions themselves were broken down into five categories: Local, Commerce, Navigation, Information, and Command. Assistant won in all but one of these categories, with Siri taking the edge only when it comes to commands.

Google Home has the edge in four out of the five categories but falls short of Siri in the Command category. HomePod’s lead in this category may come from the fact that the HomePod will pass on full SiriKit requests like those regarding messaging, lists, and basically anything other than music to the iOS device paired to the speaker. Siri on iPhone has deep integration with email, calendar, messaging, and other areas of focus in our Command category. Our question set also contains a fair amount of music-related queries, which HomePod specializes in.

What’s especially interesting in all of this, however, is how these numbers compare to last year. The same test was performed this time last year, and everyone has gotten a lot better in that time. In 2017, Assistant went through this test with only 81% correct where Alexa was at 64%. Siri wasn’t on the HomePod at the time, but as of February, the speaker ran this test with a dismal 52% accuracy.

Clearly, Google’s competitors are rapidly closing the gap between their products and Assistant. Siri especially has seen tremendous growth here and that’s certainly commendable. In the meantime, though, Google Assistant has the edge, and consumers know it as sales show.