Chrome wants to add AI features that help make common tasks easier

Abner Li | May 8 2024 - 3:25 pm PT

To date, Google Chrome has launched three generative AI features and more that integrate Gemini are teased for this year. An interview today provides a look at the development process.

“We’d been thinking about how to bring AI technology to the browser to make the typical actions you do every day — using tabs, using Search, writing in forms, reading webpages — a little easier,” says Chrome engineering director Adriana Porter Felt. “We solicited ideas from all over the Chrome team.”

“When we think about the future of Chrome,” Adriana says, “we just keep thinking, ‘How can we be as helpful as possible?’”

Of those four common tasks, the current set of features don’t yet address “using Search” or “reading webpages.” The latter might summarize articles, while the former could integrate with SGE. The Chrome Omnibox did just add a @Gemini shortcut to quickly access gemini.google.com.

The development process of LLM-powered features involves taking the foundational model and “teach[ing] it what we want it to do for our example use cases, and then we evaluate how it works against many different types of user scenarios.”

Google launched “Organize Similar Tabs” in January, with the tab organizer using emoji to make it easier to scan those groupings. One concern was making sure Chrome didn’t pick something inappropriate:

For example, If you’re planning a celebration of life, Adriana explains, they don’t want Chrome to show you a skull and crossbones. So, with help from Google’s emoji team, they decided to map out what kinds of tab group categories were safe for broad use. “Travel, animals, places, nature — these kinds of things are great use cases for emoji, so we know the auto-tab organizer has a good chance of getting it right by only drawing from those categories,” she says.

Meanwhile, the theme creator does not accept open prompts as originally planned after the team found that it was “actually kind of difficult to come up with a prompt for this.” Instead, like on Android, the tool offers various dropdown menus: Subject, Style, and Mood. In this case, “narrower options” make for better results, while also making sure it’s not used as a general text-to-image generator.

Finally, Help me write first gathers context about the webpage: “It can see you want to write a restaurant review and adjust for that versus helping you fill out a form or sell something.”