Many of the Google Lens features announced at I/O 2019 have launched. This includes the redesign of Google’s visual search feature with Dining and Translate filters, and an AR experience. You can now try out the Google Lens AR experiences below.
The latest Google Lens iteration widely rolled out to Android and iOS in late May. New features include automatic “Translate” that overlays text in your desired language over the original work to maintain context, and “Dining” to point at restaurant menus to get pictures and reviews from Google Maps.
Other filters include “Text” for optical character recognition (OCR) and fast copying/pasting from the real world, and “Shopping” for barcodes, clothing, furniture, and more. The default search mode is “Auto,” with Google Lens AR features accessible from here.
Google wants to make Lens the gateway to online content by having it recognize objects in the real world and then offering more information. For example, pointing Lens at paintings can bring up more about the work from select museums. Scanning an NBA team’s logo during a live game loads live scores.
What makes it augmented reality is how Google blends videos and animations with the physical environment by often keeping the object that launched the experience — and your surroundings — in the live frame. They are officially known as “Living Surfaces,” with the following real examples:
Google did an interesting promotion with Netflix in the New York Times for Stranger Things 3 last month. The three ads — previously only available in the print edition — above will launch a themed interaction. (Open the images one at a time in fullscreen.) Once recognized, a pill noting the ad appears at the very top of your screen, with a volume mute control in the bottom-right.
All the examples below feature the Google Lens camera logo to signal that pointing your phone will load an experience. The Stranger Things experience features a link to goo.gle/Lens and themed “Unlock something strange” tagline. The movie poster print is particularly promising, while the last is a shopping demo.