Google Search’s futuristic new tools will be able to identify and filter products in the real world [Video]

Ben Schoon | May 11 2022 - 10:28 am PT

Google Search is a powerful tool that harnesses the power of everything Google knows and uses it to give you more information about the world around you. Today at Google I/O 2022, Google has announced an expansion of Lens called “scene exploration” that will let the product “explore” your surroundings, identifying products and even filtering them in real time, as well as more enhancements to multisearch.

“Multisearch” is considered as one of Google’s biggest updates to Search in quite some time, offering the ability to use a picture as a starting point for a search and then adding more specifying details after.

At I/O 2022, Google has announced an expansion to “multisearch” which will allow this idea to work for finding local products. Adding “near me” to a query will help you find products at local stores, or food from nearby restaurants, based on the picture you upload.

For example, say you see a colorful dish online you’d like to try – but you don’t know what’s in it, or what it’s called. When you use multisearch to find it near you, Google scans millions of images and reviews posted on web pages, and from our community of Maps contributors, to find results about nearby spots that offer the dish so you can go enjoy it for yourself.

One of Google’s examples of this functionality, as seen below, uses an image of a kitchen sink faucet to start the search, with “replacement hose” and “near me” narrowing down the search to local stores. Another example finds a specific dish that’s being served, and lists local restaurants that offer it.

Google says that this will start rolling out later this year in English.

“Scene exploration” in Google Lens will allow your phone to identify multiple products or objects around you, simply by moving your camera around the scene to gather information. Lens will automatically recognize multiple objects and offer insights.

The use case Google details for this is shopping for a specific chocolate bar within a store. On the shelf, there might be dozens of options, which Lens can recognize and help you filter. As seen in the demo below, “scene exploration” is able to recognize each individual chocolate bar and show its online review ratings. From there, Google Lens can help you filter options based on the highest ratings or type of chocolate, and even look for a “nut-free” option.

On stage during I/O, Google called the tool a “Ctrl + F” for the real world, also bringing up examples of trying to find wine in a store from a black-owned winery, or a social worker sorting through donations with more ease.

This futuristic ability seems like it will be handy in local shopping trips, but it’s not available just yet. Google says that “scene exploration” in Lens will be coming “in the future.” There’s no strict timeline as to when it might arrive.