Google opens Assistant Smart Displays to interactive education and storytelling apps

At I/O 2019, Google announced the Interactive Canvas framework to bring full-screen visuals and animations to what were previously voice-only Actions. After making a big push for games on Smart Displays, Google is now letting developers build interactive education and storytelling apps.

The Interactive Canvas API lets developers build touch and voice apps using HTML, CSS, and Javascript. Example educational use cases include cooking and teaching, while storytelling is like games but somewhat more static. 

In terms of discovering these new experiences, Google will be creating new Education and Storytelling “hubs.” Joining games, they will appear on the homescreen of Nest Hubs or can be accessed with “Hey Google, teach me something new.”  

This opens a carousel with several categories: Learn a new language, Pick up a new skill, or Discover the world. This in turn shows cards of YouTube videos and app actions that you can launch. 

Meanwhile, Google is making it easier to buy things on Smart Displays. Voice Match can be used to authorize a payment, but next year devices will support entering your card’s CVC on-display.

Other announcements today as part of Google Assistant Developer Day include:

• Two new English voices for use in Actions with improved prosody model to make Assistant sound more natural
• Actions Testing API for running “end to end tests in an isolated preview environment, run regression tests, and add continuous testing to your arsenal.” This API is hitting general availability soon.
• Dialogflow migration tool
• Relaunch of Assistant links to “deliver rich Google Assistant experiences in their websites as well as deep links to their Google Assistant integrations from anywhere on the web.”