Pixel Tablet will let you ‘Look and Sign’ to Assistant

Following rumors of a Pixel Tablet 2 in the works, Google appears to be readying a new “Look and Sign” feature to enable a new way to interact with the Google Assistant.

About APK Insight: In this “APK Insight” post, we’ve decompiled the latest version of an application that Google uploaded to the Play Store. When we decompile these files (called APKs, in the case of Android apps), we’re able to see various lines of code within that hint at possible future features. Keep in mind that Google may or may not ever ship these features, and our interpretation of what they are may be imperfect. We’ll try to enable those that are closer to being finished, however, to show you how they’ll look in the case that they do ship. With that in mind, read on.

One of the features of Nest Hub Max that has yet to make its way to the Pixel Tablet is “Look and Talk.” Humorously referred to internally as “blue steel” or “LnT,” Look and Talk uses the display’s built-in camera to (locally) recognize when someone is both looking at and speaking to the device. We’ve previously found signs of Look and Talk in development for the Pixel Tablet, but six months later, it has yet to launch.

In the latest beta update to the Google Search app, version 15.15, a few in-app texts related to Look and Talk were removed and replaced by new ones labeled “LnS.” We forcibly enabled the work-in-progress feature, which appears alongside Look and Talk in the “Assistant on Hub Mode” settings.

What especially caught our interest is that Google has chosen to “REDACT” these texts to hide something. See for yourself:

Use REDACT with LNS

You can use REDACT to communicate with Assistant when your tablet is docked.

How it works:

• Your device relies on camera sensing and analyzes video of you or others in your home to determine if anyone wants to activate Assistant.
• Agreeing to this setting lets Assistant respond to REDACT.
• Assistant may activate when someone didn’t intend it to, like if they REDACT.
• Change these options anytime in your Assistant settings.

With a bit of digging into the code, we’ve found that “LNS” stands for “Look and Sign.” Reading between the lines, we have two ideas of what Look and Sign might do on the Pixel Tablet. The less ambitious idea is that Google may introduce a hand gesture you can use instead of saying “Hey Google.” This could be something like pointing at the screen, giving a thumbs up, or flipping the bird (not likely) – anything that more clearly indicates that you’re trying to talk to the Assistant.

Or, if we take “sign” more literally, Look and Sign may allow someone who is deaf/mute to interact with the Google Assistant using sign language. This would require a significant amount of processing, but Google has previously shown examples of using machine learning to detect and interpret sign languages.

In either case, it’s somewhat interesting that Google is working on a new feature that focuses on the Assistant. In recent months, the Google Assistant has been gradually replaced by Gemini everywhere but the smart home. Even the Pixel Tablet, which seems to be on track to gain Gemini support soon, will differentiate between Assistant and Gemini depending on whether the device is docked.

With rumors of a Pixel Tablet 2 in the works, adding a novel Google Assistant feature like Look and Sign suggests that the Assistant may be here to stay, at least in the smart home.

What do you think Look and Sign could mean for the Google Assistant on Pixel Tablet? Let us know in the comments below.

Thanks to JEB Decompiler, from which some APK Insight teardowns benefit.