It’s been clear for a while that Google is working to let Gemini control your Android phone, and we now have some details on how this “screen automation” will work.
About APK Insight: In this “APK Insight” post, we’ve decompiled the latest version of an application that Google uploaded to the Play Store. When we decompile these files (called APKs, in the case of Android apps), we’re able to see various lines of code within that hint at possible future features. Keep in mind that Google may or may not ever ship these features, and our interpretation of what they are may be imperfect. We’ll try to enable those that are closer to being finished, however, to show you how they’ll look in case that they do ship. With that in mind, read on.
The Google app 17.4 beta contains strings about a “Get tasks done with Gemini” feature. This Labs capability is codenamed “bonobo” and the introductory strings explain how “Gemini can help with tasks, like placing orders or booking rides, using screen automation on certain apps on your device.”
<string name=”assistant_robin_bonobo_fre_title”>Get tasks done with Gemini</string>
This control is also referred to as “screen automation,” with Android 16 QPR3 laying the groundwork. It will be available in “certain apps.”
Google warns that “Gemini can make mistakes” and “You’re responsible for what it does on your behalf, so supervise it closely.” At any time, you will be able to stop the agent and manually take over the task.
On the privacy front:
- “When Gemini interacts with an app, screenshots are reviewed by trained reviewers and used to improve Google services if Keep Activity is on.”
- “Don’t enter login or payment information into Gemini chats. Avoid using screen automation for emergencies or tasks involving sensitive information.”
Also of note in this beta are strings about a “Likeness” feature or integration codenamed “wasabi.” Notably, that’s how Android XR refers to 3D avatars that are currently used for Google Meet calls. One string suggests being able to access it for a prompt.
<string name=”assistant_robin_wasabi_ready”>Likeness ready</string>
<string name=”assistant_robin_wasabi_retake”>Retake</string>
<string name=”assistant_robin_wasabi_privacy_notice”>Your likeness can only be used by you</string>
FTC: We use income earning auto affiliate links. More.
Comments