This is what Soli ‘sees’ when you perform gestures on the Pixel 4

This month’s Pixel Feature Drop introduced a new Motion Sense gesture to play/pause music. The Google ATAP team today detailed the road to getting radar on the Pixel 4 and how Soli gestures work.

Soli’s short-range radar has two primary user-facing capabilities. The first is gesture recognition in the form of tapping or swiping above the Pixel 4 to control music, answer/decline calls, and end/snooze alarms. More subtle is detecting when you’re physically near or reaching for your phone to turn on the screen and ready face unlock.

Google starts by noting how Soli has a “fundamentally different sensing paradigm based on motion, rather than spatial structure” compared to traditional radar on planes and cars that’s designed for detecting large, rigid, and distant objects. This allows for a significantly smaller antenna ray that fits on a 5mm x 6.5mm x 0.873mm chip package.

Meanwhile, the machine learning algorithms behind Soli “do not require forming a well-defined image of a target’s spatial structure,” which means there’s “no distinguishable images of a person’s body or face are generated or used” by Google.

Soli relies on processing temporal changes in the received signal in order to detect and resolve subtle motions.

Google today shared animations that show how Soli interprets motion and gestures. The system accounts for unwanted interference, including audio vibrations when music is playing.

• The vertical axis of each image represents range, or radial distance, from the sensor, increasing from top to bottom. The horizontal axis represents velocity toward or away from the sensor, with zero at the center, negative velocities corresponding to approaching targets on the left, and positive velocities corresponding to receding targets on the right.
• Energy received by the radar is mapped into these range-velocity dimensions and represented by the intensity of each pixel. Thus, strongly reflective targets tend to be brighter relative to the surrounding noise floor compared to weakly reflective targets. The distribution and trajectory of energy within these range-velocity mappings show clear differences for a person walking, reaching, and swiping over the device.

Left: Presence — Person walking toward the device. Middle: Reach — Person reaching toward the device. Right: Swipe — Person swiping over the device.