Gemini app rolling out ‘Utilities’ Extension for phone control: Here’s everything it can do

Abner Li | Nov 7 2024 - 8:30 am PT

As previewed back at I/O 2024 in May, Google is beginning to roll out the Gemini Utilities Extension that can “control your Android mobile device & apps.”

The Utilities Extension allows Gemini to “manage your alarms and timers, open apps, control your media, and more.” Voice and text prompts are accepted. It replaces the current Google Assistant fallback for a more native experience.

Google is “gradually releasing this extension, so it might not be available to you just yet” for “prompts in English only” at the moment. Like WhatsApp, Google Home, Phone, and Messages last week, we’re not seeing this widely available yet.

Request multiple actions at once
Open websites, apps & settings
Check & control your device features
Take photos & screenshots
Control your media playback
Browse or search for screenshots in Pixel Screenshots
Set & manage alarms & timers in your clock app

Meanwhile, some of these features work when using Gemini from your lockscreen:

Set and silence alarms
Set and stop timers
Turn the following features on or off: Flashlight, Bluetooth, Do Not Disturb, and Battery Saver
Check your device’s volume and battery level
Power off or restart your device
Take a photo and screenshot
Control media, like pause or replay a video

Here’s what the Gemini Utilities Extension can do and prompts that work:

Request multiple actions at once

You can ask the Gemini mobile app to perform multiple actions in one prompt.

Turn my media volume and notification volume to [percentage] and turn on battery saver.
Take a photo with a 10s timer

Open websites, apps & settings

You can ask the Gemini mobile app to open a website, your apps, and settings menus.

Open [URL name]
Open [app name]
Open [name of settings] settings

Check & control your device features

You can ask the Gemini mobile app to do the following actions on your device:

Turn [on, off] flashlight
Turn [on, off] battery saver
Tell me the volume
Volume [percentage]
Turn [up, down] volume
Decrease brightness to [percentage]
What’s my battery level?
Power off device
Restart my device

Take photos & screenshots

You can ask the Gemini mobile app to take a photo or screenshot.

Take a selfie
Take a picture in [seconds]
Take a screenshot

Control your media playback

You can ask the Gemini mobile app to control media playback on your device.

Pause
Pause the [music, video]
Stop the [music, video]
Resume
Continue playing
Replay
Restart [music, video]
Skip
Next
Next [song, video]
Previous
Play previous
Give this [song, video] a thumbs up

Browse or search for screenshots in Pixel Screenshots

You can ask the Gemini mobile app to find a screenshot for you in the Pixel Screenshots app.

This was announced with the October Pixel Drop.

Search for [item, object, content type] in Pixel Screenshots
Show me my [collection name] in Pixel Screenshots
Check Pixel Screenshots. [your query].

Set & manage alarms & timers in your clock app

If the Clock app by Google is enabled on your device, the Gemini mobile app uses it over any other clock app.
For third-party clock apps, the Gemini mobile app can only set alarms and timers. Other actions aren’t supported.

Set & manage alarms & timers based on your conversation

Set an alarm so I can wake up for [event, activity].
I need to focus on [activity, task] for [length of time]. Set a timer for this.
Show my alarms.
- After Gemini’s response: Delete the [time] alarm.
When does [activity, event] start?
- After Gemini’s response: Set an alarm for [hours] hours earlier.

Set & manage your alarms