Sundar Pichai addresses Gemini’s ‘unacceptable’ errors and ‘at scale’ fixes

Sundar Pichai on Tuesday sent an internal memo reiterating how Google “got it wrong” with Gemini responses and image generation. The CEO told employees that they have to “focus on what matters most: building helpful products that are deserving of our users’ trust.” 

I want to address the recent issues with problematic text and image responses in the Gemini app (formerly Bard). I know that some of its responses have offended our users and shown bias — to be clear, that’s completely unacceptable and we got it wrong.

The email to employees was published by Semafor and The Verge in full. Google’s public-facing response on Friday from the SVP of Search said it “did not want Gemini to refuse to create images of any particular group,” nor have it “create inaccurate historical — or any other — images.” 

Sundar Pichai echoed that and reiterated Google’s core mission:

Our mission to organize the world’s information and make it universally accessible and useful is sacrosanct. We’ve always sought to give users helpful, accurate, and unbiased information in our products. That’s why people trust them. This has to be our approach for all our products, including our emerging Al products.

We’ll be driving a clear set of actions, including structural changes, updated product guidelines, improved launch processes, robust evals and red-teaming, and technical recommendations. We are looking across all of this and will make the necessary changes.

Last week, the company identified two issues with Gemini image generation:

1. “…our tuning to ensure that Gemini showed a range of people failed to account for cases that should clearly not show a range.”
2. “…over time, the model became way more cautious than we intended and refused to answer certain prompts entirely — wrongly interpreting some very anodyne prompts as sensitive.”

Pichai says the company has “been working around the clock to address these issues.” Google pulled Gemini’s ability to generate images of people last week until it can “significantly” improve it.

We’re already seeing a substantial improvement on a wide range of prompts. No Al is perfect, especially at this emerging stage of the industry’s development, but we know the bar is high for us and we will keep at it for however long it takes. And we’ll review what happened and make sure we fix it at scale.

It will be interesting to see what that “at scale” fix entails, especially on the technical front, while another thing to watch for is whether the “structural changes” will result in Google slowing down the pace of launches to allow for more QA. How Google details and quantifies the improvements will also be notable.