Pixel 5’s new ‘Hold for Me’ uses Google Duplex to distinguish between recorded and live humans

One of the most impressive software features to debut with the Pixel 4a 5G and Pixel 5 is “Hold for Me.” The Google Assistant will stay on the line until an actual human being comes on for you to directly interact with.

When placed on hold after calling a toll-free number, Assistant can be invoked to wait on your behalf. There’s a large “Don’t hang up” prompt with a “Return to call” option available at any time. Meanwhile, a text transcript appears at the bottom of the screen detailing what’s happening.

When a human finally gets on the other line, your phone will ring/vibrate and you can tap the green button. To make sure the other person doesn’t hang up, Google will ask them to hold for a moment to give you a chance to return.

Behind-the-scenes, Hold for Me is powered by Google Duplex. This “AI system for accomplishing real-world tasks over the phone” is already used to make appointments on your behalf, and check store hours.

In this application, Google “not only recognizes hold music but also understands the difference between a recorded message (like ‘Hello, thank you for waiting’) and a representative on the line.” The company says this is not possible with “simple algorithms” since “every business’s hold loop is different.”

The audio processing is done privately on-device and does not require a Wi-Fi/data connection. 

…no audio from the call will be shared with Google or saved to your Google account unless you explicitly decide to share it and help improve the feature. When you return to the call after Google Assistant was on hold for you, audio stops being processed altogether.

Google worked with Dell, United, and other companies, and used customer support representative studies to “design these interactions and make the feature as helpful as possible to the people on both sides of the call.”

Hold for Me must first be enabled in settings and activated with each call. It is still in preview and initially only available in the US. More users will have access to it “over the coming months.”