Google Photos using ML to predict depth and create new 3D ‘Cinematic photos’

Abner Li | Dec 15 2020 - 9:00 am PT

Besides organizing and making your library searchable, Google Photos often generates videos, albums, and other fun works. Google is now creating 3D “Cinematic photos,” while more collage designs and Memories are rolling out.

With Cinematic photos, Google uses “machine learning to predict an image’s depth and produce a 3D representation of the scene—even if the original image doesn’t include depth information from the camera.” Few cameras have depth sensors, but various Google products — including ARCore and the Pixel’s Camera app for Portrait Mode — can calculate it from a flat input.

In Photos, this works to separate the background from the subject (and foreground). The latter is enlarged to take up most of the vertical screen as part of a “smooth panning effect.” It attempts to replicate what you were seeing when originally taking the picture.

Google will create Cinematic photos for you and surface them in the “Recent highlights” section at the top of the app, as well as through a new Search category. Like other creations, they can be saved in your library and shareable as video clips. The company notes how it’s still improving the format and taking in-app feedback.

Meanwhile, Google is adding more collage designs that feature “richer, artistically designed layouts populated and stylized using AI.” Pictures with similar colors will be paired together as a matching font, frame, and background is applied. Overall, the goal is to provide a “more cohesive look.”

The Memories carousel currently highlights what happened in previous years. It will now start surfacing the “most important people in your life.” Google will soon also feature your favorite things and activities, like sunsets, baking, and hiking.

These new features are rolling out over the next month with the latest app version.