Latest test shows Google Assistant as the clear leader in smartphone digital assistants

The latest digital assistant test from Loup Ventures is out today, providing a comprehensive look at how the top four players compare. Just like last year’s test, Google Assistant comes out as the clear winner above Apple’s Siri, Microsoft Cortana, and Amazon Alexa in almost all categories…

It’s worth noting that this particular test only covers the abilities of smartphone assistants, not smart speakers (although Loup does test those, too). The test runs Assistant on a Pixel and Siri on an iPhone, but for Microsoft Cortana and Amazon Alexa, the test involves using third-party apps running on an iPhone. It goes without saying the latter are starting with a disadvantage.

(A couple other studies earlier this year, and another from Loup, might give you a better idea of how the smart assistants compare on smart speakers. Spoiler: Google Assistant still wins.)

The most important numbers to look at for this test are in the “Answered Correctly” column. Given 800 queries (slightly modified since last year’s test, “so that [the test] remains exhaustive”), Google Assistant is up to 85.5% from last year’s 74.8%. Its competitors are also showing gains, most notably Apple’s Siri which has jumped from a failing 66.1% grade to 78.5%.

The test is divided into 5 categories: Local, Commerce, Navigation, Information, and Command. Breaking out the correctly answered questions by these categories shows Google in the clear lead in all but one. At least per this test, Siri has better baked-in phone control than Assistant.

Siri’s lead over the Assistant in this category is odd, given they are both baked into the OS of the phone rather than living on a 3rd party app (as Cortana and Alexa do). We found Siri to be slightly more helpful and versatile (responding to more flexible language) in controlling your phone, smart home, music, etc.

Perhaps one of the most fascinating results of this study, it seems Google Assistant has managed to nab a perfect score in interpretation of the 800 queries thrown at it. It already had a strong 99% last year, a score Siri has only just now caught up with. Cortana and Alexa are both lagging behind at 98%.

9to5Google’s Take

This test is in line with what we’ve been seeing for a few years now: Google is the clear overall leader in digital assistants.

The biggest and most noticeable difference between Assistant and Siri is the “Information” category. You can tell when using Assistant that it gets an impressive 93% of those kinds of questions right, and likewise with Siri’s 70%. That 23 percentage points is the difference between giving the assistant a shot to provide a quick answer and just Googling it yourself.

Check out 9to5Mac on YouTube for more Apple news: