Google Camera 6.2 adds kiss detection to Pixel 3’s Photobooth mode

With the Pixel 3, the Google Camera app gained a number of AI-powered features like Night Sight, Top Shot, and Photobooth. The latter is essentially Google Clips running on a phone, with the mode adding automatic kiss detection.

Photobooth is another mode accessible from the “More” tab in Google Camera that automatically takes pictures for users. It leverages machine learning to find the best moments and capture them without users having to manually press the shutter every time.

It’s optimized for the Pixel 3’s dual front-facing cameras, and looks for solo, couple, and group selfies. Google Camera 6.2 adds new kiss detection that will automatically take a shot when a kiss is in frame. Google announced this new feature in a Google AI blog post describing how the feature works and the associated challenges. Much of the work behind Google Clips was leveraged for the Pixel 3.

Photobooth features two on-device models for understanding what is a good selfie. The first looks for facial expressions and the second detects when people kiss.

We worked with photographers to identify five key expressions that should trigger capture: smiles, tongue-out, kissy/duck face, puffy-cheeks, and surprise. We then trained a neural network to classify these expressions. The kiss detection model used by Photobooth is a variation of the Image Content Model (ICM) trained for Google Clips, fine tuned specifically to focus on kissing.

Various frames are then scored, with a “final image quality score used for triggering the shutter is computed by a weighted combination of the attention based facial expression score and the kiss score.”

The frame score considers both facial expression quality and the kiss score. As the kiss detection model operates on the entire frame, its output can be used directly as a full-frame score value for kissing. The face expressions model outputs a score for each identified expression.

Google Camera 6.2 should be widely rolled out via the Play Store, with Photobooth exclusive to the Pixel 3 and Pixel 3 XL.